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Preface 



Up close, smooth things look flat — the picture behind differential calculus. In 
mathematical language, we can approximate smoothly varying functions by 
linear functions. In calculus of several variables, the resulting linear functions 
can be complicated: you need to study linear algebra. 

Problems appear throughout the text, which you must learn to solve. They 
often provide vital results used in the course. Most of these problems have 
hints, particularly the more important ones. There are also review problems at 
the end of each section, and you should try to solve a few from each section. 
Try to solve each problem first before looking up the hint. Never use decimal 
approximations (for instance, from a calculator) on any problem, except to 
check your work; many problems are very sensitive to small errors and must 
be worked out precisely. Whenever ridiculously large numbers appear in the 
statement of a problem, this is a hint that they must play little or no role in 
the solution. 

The prerequisites for this course are basic arithmetic and elementary algebra, 
typically learned in high school, and some comfort and facility with proofs, 
particularly using mathematical induction. You can't prove that all men are 
wearing hats just by pointing out one example of a man in a hat; most proofs 
require an argument, and not just examples. Polya [2] and Solow [3] explain 
induction and provide help with proofs. Bretscher [1] and Strang [5] are excellent 
introductory textbooks of linear algebra. 

For teachers 

These notes are drawn from lectures given at University College Cork in the 
spring of 2006, for a first year introduction to linear algebra. The course aims 
for a complete proof of the spectral theorem, but with two gaps: (1) the proof 
for real symmetric matrices relies on the minimum principle, so requires the 
existence of a minimum of any quadratic function on the sphere; (2) the proof 
for complex self- adjoint matrices requires the fundamental theorem of algebra to 
prove that complex matrices have eigenvalues. We fill these gaps in appendices, 
but the students are not expected to work through the more difficult material 
provided in these appendices. There are a number of proofs that are not quite 
complete, giving only the idea behind the proof. In each case, giving a complete 
proof just requires adding in summation notation, which students often find 
confusing. I teach a small selection of the proofs. 
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I aim to make each chapter one lecture of material, but that hasn't always 
worked out. With that aim in mind, the chapters are unusually small, but 
students should find them easier to grasp. The book presents just the material 
required to reach the spectral theorem for self-adjoint matrices. This gives the 
course a natural focal point. 

We first approach determinants by direct calculation, shying away from 
proofs via permutations, and from Cramer's rule and cofactor inversion, which 
are computationally infeasible. I hope that students learn how to compute with 
simple examples by hand, and then learn the theory. I ignored purely numerical 
topics and paid no attention to computational efficiency 
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Matrix Calculations 



Chapter 1 



Solving Linear Equations 



In this chapter, we learn how to solve systems of linear equations by a simple recipe, 
suitable for a computer. 



Elimination 

Consider equations 

-6 - x 3 + x 2 = 
3xi + 7x 2 + 4x 3 = 9 
3xi + 5x 2 + 8x 3 = 3. 

They are linear because they are sums of constants and constant multiples of 
variables. How can we solve them (or teach a computer to solve them)? To 
solve means to find values for each of the variables x\,x 2 and X3 satisfying all 
three of the equations. 

Preliminaries 

a. Line up the variables: 

x 2 - x 3 = 6 
3xi + 7x 2 +4x 3 = 9 
3xi + 5x2 + 8x3 = 3 

All of the xi's are in the same column, etc. and all constants on the right 
hand side. 

b. Drop the variables and equals signs, just writing the numbers. 
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\3 5 


8 3j 



This saves rewriting the variables at each step. We put brackets around 
for decoration. 
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c. Draw a box around the entry in the top left corner, and call that entry 
the pivot. 

9 

3; 
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Forward elimination 

(1) If the pivot is zero, then swap rows with a lower row to get the pivot to 
be nonzero. This gives 
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(Going back to the linear equations we started with, we are swapping the 
order in which we write them down.) If you can't find any row to swap 
with (because every lower row also has a zero in the pivot column), then 
move pivot one step to the right — > and repeat step (1). 

(2) Add whatever multiples of the pivot row you need to each lower row, in 
order to kill off every entry under the pivot. ("Kill off" means "make into 
0"). This requires us to add — (row 1) to row 3 to kill off the 3 under the 
pivot, giving 

3 7 4 9 \ 
1-16- 

-2 4 -6y 

(Going back to the linear equations, we are adding equations together which 
doesn't change the answers — we could reverse this step by subtracting 
again.) 

(3) Make a new pivot one step down and to the right: \. 
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and start again at step (1). 

In our example, our next pivot, 1, must kill everything beneath it: —2. So we 
add 2 (row 2) to (row 3), giving 

/ 3 7 4 9\ 

[T] -1 6 . 
v 2 6/ 




Figure 1.1: Forward elimination on a large matrix. The shaded boxes are nonzero entries, while 
zero entries are left blank. The pivots are outlined. You can see the first few steps, and then a 
step somewhere in the middle of the calculation, and then the final result. 



We are done with that pivot. Move 
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Forward elimination is done. Let's turn the numbers back into equations, to 
see what we have: 



3a;i 



+7x 2 



X2 



-X3 

2x 3 



= 9 
= 6 
= 6 



Look from the bottom equation up: e 
of later variables. 

1.1 Apply forward elimination to 

/0 
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1.2 Apply forward elimination to 
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1.3 Apply forward elimination to 

( l 1 
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pivot solves for one variable in terms 
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Figure 1.2: Back substitution on the large matrix from figure 1.1. You can see the first few steps, 
and then a step somewhere in the middle of the calculation, and then the final result. You can see 
the pivots turn into l's. 



Back Substitution 

Starting at the last pivot, and working up: 

a. Rescale the entire row to turn the pivot into a 1. 

b. Add whatever multiples of the pivot row you need to each higher row, in 
order to kill off every entry above the pivot. 

Applied to our example: 
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Scale row 3 by \ : 
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Add row 3 to row 2, —4 (row 


3) to row 
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Add —7 (row 2) to row 1. 
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Scale row 1 by |. 
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Done. Turn back into equations: 



x\ = - 
x 2 = 9 
X3 = 3 
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Forward elimination and back substitution together are called Gauss-Jordan 
elimination or just elimination. (Forward elimination is often called Gaussian 
elimination.) Forward elimination already shows us what is going to happen: 
which variables are solved for in terms of which other variables. So for answering 
most questions, we usually only need to carry out forward elimination, without 
back substitution. 

Examples 

More than one solution: 

xi + x 2 + x 3 + x i = 7 
%\ + 2x 3 =1 
x 2 + x 3 =0 



Write down the numbers: 
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Kill everything under the pivot: add — (row 1) to row 2. 

(l 




Done with that pivot; move 
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Kill: add row 2 to row 3: 
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Move \,. Forward elimination is done. Let's look at where the pivots lie: 
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Let's turn back into equations: 

35 1 +X2 +x 3 +x 4 = 7 
-352 +X3 —X4=—6 
2X3 —X4=— 6 



Look: each pivot solves for one variable, in terms of later variables. There was 
never any pivot in the x 4 column, so X4 is a free variable: X4 can take on any 
value, and then we just use each pivot to solve for the other variables, bottom 
up. 

1.4 Back substitute to find the values of x\, X2, X3 in terms of .T4. 
No solutions: consider the equations 
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2xi 
Axi 



£2 + 2x3= 1 
£3=0 



X2- 



■ 3X2 + 5Z3 = 1. 



Forward eliminate: 
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Add — 2(row 1) to row 2, — 4(row 1) to row 3. 



Move the pivot 



Add —(row 2) to row 3 
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Move the pivot \ . 



Move the pivot — K 
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Turn back into equations: 

a:i + x 2 + 2 x 3 = 1 
— X2 - 3x 3 = -2 
= -1. 

You can't solve these equations: can't equal —1. So you can't solve the 
original equations either: there are no solutions. Two lessons that save you time 
and effort: 

a. If a pivot appears in the constants' column, then there are no solutions. 

b. You don't need to back substitute for this problem; forward elimination 
already tells you if there are any solutions. 

Summary 

We can turn linear equations into a box of numbers. Start a pivot at the top left 
corner, swap rows if needed, move — > if swapping won't work, kill off everything 
under the pivot, and then make a new pivot \ from the last one. After forward 
elimination, we will say that the resulting equations are in echelon form (often 
called row echelon form) . 

The echelon form equations have the same solutions as the original equations. 
Each column except the last (the column of constants) represents a variable. 
Each pivot solves for one variable in terms of later variables (each pivot "binds" 
a variable, so that the variable is not free). The original equations have no 
solutions just when the echelon equations have a pivot in the column of constants. 
Otherwise there are solutions, and any pivotless column (besides the column 
of constants) gives a free variable (a variable whose value is not fixed by the 
equations). The value of any free variable can be picked as we like. So if 
there are solutions, there is either only one solution (no free variables), or 
there are infinitely many solutions (free variables). Setting free variables to 
different values gives different solutions. The number of pivots is called the 



rank. Forward elimination makes the pattern of pivots clear; often we don't 
need to back substitute. 

We often encounter systems of linear equations for which all of the constants 
are zero (the "right hand sides"). When this happens, to save time we won't 
write out a column of constants, since the constants would just remain zero all 
the way through forward elimination and back substitution. 

1.5 Use elimination to solve the linear equations 

2 x 2 + x 3 = 1 
Axi - x 2 + x 3 = 2 
4 xi + 3 x 2 + 3 x 3 = 4 



Review problems 

1.6 Apply forward elimination to 
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1.7 Apply forward elimination to 
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1.8 Apply forward elimination to 
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1.9 Apply forward elimination to 



1.10 Apply forward elimination to 
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1.11 Apply forward elimination to 

ll 3 2 
2 5 4 
\3 8 6 

1.12 Apply back substitution to the result of problem 1.3 on page 5. 

1.13 Apply back substitution to 
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1.14 Apply back substitution to 
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1.15 Apply back substitution 


to 








1 




G 


3 









1.16 Apply back substitution 


to 





/3 



Vo 



2\ 

-1 

2 

2/ 



1.17 Use elimination to solve the linear equations 

— X\ + 2 x 2 + x 3 + x A — 1 
—x\ + 2 X2 + 2 x 3 + X4 = 
X3 + 2 xa = 

.T4 = 2 



1.18 Use elimination to solve the linear equations 

x\ + 2 £2 + 3 £3 + 4 £4 = 5 
2xi + 5x 2 + 7x 3 + llx 4 = 12 
£2 + £3 + 4 £4 = 3 



1.19 Use elimination to solve the linear equations 

— 2 x\ + x 2 + x 3 + x 4 = 
x\ — 2 x 2 + x 3 + x 4 = 
X\ + x 2 — 2 X3 + x 4 = 
11+12+13-214 = 



1.20 Write down the simplest example you can to show that adding one to 
each entry in a row can change the answers to the linear equations. So adding 
numbers to rows is not allowed. 



1.21 Write down the simplest systems of linear equations you can come up 
with that have 

a. One solution. 

b. No solutions. 

c. Infinitely many solutions. 

1.22 If all of the constants in some linear equations are zeros, must the equations 
have a solution? 



1.23 Draw the two lines \ x\ 



j..^ ±jlo,w uwu iiiiv,o 2 -"J. ~ x 2 = —\ and 2x\ + x 2 — 3 in M 2 . In your 
drawing indicate the points which satisfy both equations. 

1.24 Which pair of equations cuts out which pair of lines? How many solutions 
does each pair of equations have? 



x\ — x 2 = 
Xi + x 2 = 1 



(1) 



X! - x 2 = 4 
-2xi + 2x 2 = 1 



(2) 



xi — x 2 = 1 
-3 x\ + 3 x 2 = —3 



(3) 



(a) 




(b) 



(c) 



1.25 Draw the two lines 2xi + x 2 — 1 and x\ — 2x 2 = 1 in the xia^-planc. 
Explain geometrically where the solution of this pair of equations lies. Carry 
out forward elimination on the pair, to obtain a new pair of equations. Draw 
the lines corresponding to each new equation. Explain why one of these lines is 
parallel to one of the axes. 



x\ — x 2 + 2 x 3 = 2 
—2 xi + 2 X2 + X3 = — 2 
—3 X\ + 3 X2 — £3 = 



(1) 



-xx - x 3 = (2) 
x\ — 2 X2 — X3 = 
2 x\ — 2 ^2 — 2 £3 = — 1 

xi + x 2 + x 3 = 1 (3) 
2a + 2*2 + x 3 = 
2;i + 2; 2 + 2:3 = -1 

-2 xi + a; 2 + 2:3 = -2 (4) 
—2 xi — X2 + 2 a; 3 = 
-4x 2 + 2x 3 = 4 

-2x2-2:3 = (5) 
-xi - x 2 - x 3 = -1 
— 3xi — 3x2 — 3x3 = 

Table 1.1: Five systems of linear equations 



1.26 Find the quadratic function y = ax 2 + bx + c which passes through the 
points (x,y) = (0,2), (1,1), (2, 6). 

1.27 Give a simple example of a system of linear equations which has a solution, 
but for which, if you alter one of the coefficients by a tiny amount (as tiny as 
you like), then there is no solution. 

1.28 If you write down just one linear equation in three variables, like 2xi + 
2^2 — 2; 3 = —1, the solutions draw out a plane. So a system of three linear 
equations draws out three different planes. The solutions of two of the equations 
lie on the intersections of the two corresponding planes. The solutions of the 
whole system are the points where all three planes intersect. Which system of 
equations in table 1.1 draws out which picture of planes from figure 1.3 on the 
following page? 




(d) (e) 

Figure 1.3: When you have three equations in three variables, each one draws a plane. Solutions 
of a pair of equations lie where their planes intersect. Solutions of all three equations lie where all 
three planes intersect. 



Chapter 2 



Matrices 



The boxes of numbers we have been writing are called matrices. Let's learn the 
arithmetic of matrices. 



Definitions 

A matrix is a finite box A of numbers, arranged in rows and columns. We write 
it as 

/ An A 12 
A21 A22 ■ ■ ■ A? 



A lq \ 



i2g 



\Ap\ Ap2 ■ ■ ■ ApqJ 

and say that A is p x q if it has p rows and q columns. If there are as many 
rows as columns, we will say that the matrix is square. 

The entry A 3 i is in row 3, column 1. If we have 10 or more rows or columns 
(which won't happen in this book), we might write A\ t i instead of An. For 
example, we can distinguish A Uj i from Ain. 

A matrix x with only one column is called a vector and written 

M 

X2 



\X n J 

The collection of all vectors with n real number entries is called R n . 
Think of R 2 as the xy-planc, writing each point as 



instead of (x, y). We draw a vector, for example the vector 



as an arrow, pointing out of the origin, with the arrow head at the point 
x = 2,y = 3 : 



15 



/ 
\ 



Figure 2.1: Echelon form: a staircase, each step down by only 1, but across to the right by maybe 
more than one. The pivots arc the steps down, and below them, in the unshaded part, are zeros. 



/ 



Similarly, think of K 3 as 3-dimensional space. 
2.1 Draw the vectors: 




We draw vectors either as dots or more often as arrows. If there are too 
many vectors, pictures of arrows can get cluttered, so we prefer to draw dots. 
Sometimes we distinguish between points (drawn as dots) and vectors (drawn 
as arrows), but algebraically they are the same objects: columns of numbers. 



Review problems 

2.2 Find points in R 3 which form the vertices of a regular 

(a) cube 

(b) octahedron 

(c) tetrahedron. 



Echelon Form 



A matrix is in echelon form if (as in figure 2.1) each row is either all zeros or 
starts with more zeros than any earlier row. The first nonzero entry of each 
row is called a pivot. 

2.3 Draw dots where the pivots are in figure 2.1. 

2.4 Give the simplest examples you can of two matrices which are not in echelon 
form, each for a different reason. 

2.5 The entries An, A22, ■ ■ ■ of a square matrix A are called the diagonal. Prove 
that every square matrix in echelon form has all pivots lying on or above the 
diagonal. 



2.6 Prove that a square matrix in echelon form has a zero row just when it is 
cither all zeroes or it has a pivot above the diagonal. 

2.7 Prove that a square matrix in echelon form has a column with no pivot 
just when it has a zero row. Thus all diagonal entries are pivots or else there is 
a zero row. 

Theorem 2.1. Forward elimination brings any matrix to echelon form, without 
altering the solutions of the associated linear equations. 

Obviously proof is by induction, but the result is clear enough, so we won't 
give a proof. 



Review problems 

2.8 In one colour, draw the locations of the pivots, and in another draw the 
"staircase" (as in figure 2.1 on the preceding page) for the matrices 




Matrices in Blocks 



If 



then write 



A = 
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and B = 
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(We will often colour various rows and columns of matrices, just to make the 
discussion easier to follow. The colours have no mathematical meaning.) 
Any matrix which has only zero entries will be written 0. 



2.9 What could (0 0) mean? 



2.10 What could (A 0) mean if 



Matrix Arithmetic 



Add matrices like: 

a=( 1 2 U=( 5 6 U +B =( i+5 2+6 V 

^3 A) \7 8 J ^3 + 7 4 + 8.y 

If two matrices have matching numbers of rows and columns, we add them 
by adding their components: (A + B)ij = Aij + Bij. Similarly for subtracting. 

2.11 Let 

Find A + B. 

When we add matrices in blocks, 

(a b) + (c d) = (a + c B + d) 

(as long as A and C have the same numbers of rows and columns and B and D 
do as well). 

2.12 Draw the vectors 

-(-i) ■"(})■ 

and the vectors and u + v. In your picture, you should see that they form the 
vertices of a parallelogram (a quadrilateral whose opposite sides are parallel). 

Multiply by numbers like: 

\ i\ [7-1 7 ■ 2\ 



7 



,3 4/ \7 • 3 7-4, 



If A is a matrix and c is a number, cA is the matrix with {cA) i - = cAij. 
Suppose that 

-(-.■)■ 

The multiples x, 2x, 3x, . . . and —x, —2x, —3a;, . . . live on a straight line through 
0: 



Matrix Multiplication 



Surprisingly, matrix multiplication is more difficult. To multiply a single row 
by a single column, just multiply entries in order, and add up: 



= 1-3 + 2-4 = 3 + 8= 11. 



Put your left hand index finger on the row, and your right hand index finger on 
the column, and as you run your left hand along, run your right hand down: 



(- -0 



As your fingers travel, you multiply the entries you hit, and add up all of the 
products. 



2.13 Multiply 



-1 



(8\ 
1 

w 



To multiply the matrices 



A = 



,B = 



5 6' 
.7 8, 



multiply any row of A by any column of B: 



1-5 + 2-7 



As your left hand finger travels along a row, and your right hand down a column, 
you produce the entry in that row and column; the second row of A times the 
first column of B gives the entry of AB in second row, first column. 



2.14 Multiply 




We write J2 k in front of an expression to mean the sum for k taking on all 
possible values for which the expression makes sense. For example, if a; is a 
vector with 3 entries, 



then J^k Xk = xi + x 2 + x 3 . 

If A is p x q and B is q x r, then AB is the p x r matrix whose entries are 
(AB) ij = j: k A ik B kj . 

Review problems 

2.15 If A is a matrix and x a vector, what constraints on dimensions need to 
be satisfied to multiply Ax? What about xAl 

2.16 
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A = 
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, B 




V 2 







c = 



-1 



D 



-1 



Compute all of the following which are defined: 

AB, AC, AD, BC, CA, CD. 

2.17 Find some 2x2 matrices A and B with no zero entries for which AB = 0. 

2.18 Find a 2 x 2 matrix A with no zero entries for which A 2 = 0. 

2.19 Suppose that we have a matrix A, so that whenever x is a vector with 
integer entries, then Ax is also a vector with integer entries. Prove that A has 
integer entries. 

2.20 A matrix is called upper triangular if all entries below the diagonal are 
zero. Prove that the product of upper triangular square matrices is upper 
triangular, and if, for example 



A- 



Al2 




A u 


Am\ 


A22 




A 2i 


A 2 n 




A 33 


A34 


A 3 „ 



(with zeroes under the diagonal) and 



B 



B\2 


#13 


B u 


B\ n ^ 


B22 


-B23 




B2n 




-B33 


B34 


B^n 



B nn J 



then 



fA n B n 



AB 



A22B22 * * 

A 33 B 33 * 



\ A nn B nn J 

2.21 Prove the analogous result for lower triangular matrices. 

Algebraic Properties of Matrix Multiplication 

2.22 If A and B matrices, and AB is defined, and c is any number, prove that 
c(AB) = (cA)B = A(cB). 

2.23 Prove that matrix multiplication is associative: (AB)C = A(BC) (and 
that if cither side is defined, then the other is, and they are equal). 

2.24 Prove that matrix multiplication is distributive: A(B + C) = AB + AC 
and (P + Q)R = PR + QR for any matrices A, B, C, P, Q, R (again if one side 
is defined, then both are and they are equal). 



Running your finger along rows and columns, you see that blocks multiply 



like: 



etc. 



AC+BD 



2.25 To make sense of this last statement, what do we need to know about the 
numbers of rows and columns of A, B, C and Dl 



Chapter 3 



Inverses of Matrices 



Just as a number has a reciprocal, some matrices have an inverse matrix. 



The Identity Matrix 



Define matrices 



h = (1) , h = 







1 







1 



(: 



1 N 
1 
l) 



The n x n matrix with l's on the diagonal and zeros everywhere else is called 
the identity matrix, and written /„. We often write it as / to be deliberately 
ambiguous about what size it is. An equivalent definition: 



3.1 What could 7 13 mean? (Careful: it has two meanings.) What does J 2 
mean? 

3.2 Prove that I A = AI = A for any matrix A. 

3.3 Suppose that B is an n x n matrix, and that AB = A for any n x n matrix 
A. Prove that B = I n . 

3.4 Suppose that B is an n x n matrix, and that BA = A for any nxn matrix 
A. Prove that B = I n . 

3.5 If A and B are two matrices and Ax — Bx for any vector x, prove that 



The columns of /„ are vectors called d, e 2 , . . . , e„. 

3.6 Consider the identity matrix I3. What are the vectors e\, e%, e%7 

3.7 The vector ej has a one in which row? And zeroes in which rows? 

3.8 If A is a matrix, prove that Ae\ is the first column of A. 




A = B. 
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3.9 If A is any matrix, prove that Aej is the j-th column of A. 
If A is a p x q matrix, by the previous exercise, 
A = (Aei Ae 2 . . . Ae q ) . 
In particular, when we multiply matrices 

AB = (ABa ABe 2 . . . ABe q ) 



(and if either side of this equation is defined, then both sides are and they are 
equal). In other words, the columns of AB are A times the columns of B. 
This next exercise is particularly vital: 

3.10 If A is a matrix, and x a vector, prove that Ax is a sum of the columns 
of A, each weighted by entries of x: 

Ax = xi (Aei) + x 2 (Ae 2 ) H \- x n (Ae n ) . 

Review problems 

3.11 True or false (if false, give a counterexample): 

a. If the second column of B is 3 times the first column, then the same is 
true of AB. 

b. Same question for rows instead of columns. 

3.12 Can you find matrices A and B so that A is 3 x 5 and B is 5 x 3, and 
AB = I? 

3.13 Prove that the rows of AB are the rows of A multiplied by B. 

3.14 The Fibonacci numbers are the numbers xo — 1, x\ — l,a; n +i = x n + x n -\. 
Write down xo,x\,x 2 ,X3 and x\. Let 




Prove that 




3.15 Let 




Draw these vectors in the plane. 



Let 





(a) The 
original 




For each matrix M = A, B, C, D, E, F draw Mi and My (in a different colour 
for each matrix), and explain in words what each matrix is "doing" (for example, 
rotating, flattening onto a line, expanding, contracting, etc.). 

3.16 The first picture in figure 3.1 is the original in the x±,X2 plane, and 
the center of the circular face is at the origin. If we pick a matrix A and set 
y = Ax, and draw the image in the 2/1,2/2 plane, which matrix below draws 
which picture? 






3.17 Can you figure out which matrices give rise to the pictures in the last 
problem, just by looking at the pictures? Assume that you known that all the 
entries of each matrix are integers between -2 and 2. 

3.18 What are the simplest examples you can find of 2 x 2 matrices A for which 
taking vectors x to Ax 

(a) contracts the plane, 

(b) dilates the plane, 

(c) dilates one direction, while contracting another, 

(d) rotates the plane by a right angle, 

(e) reflects the plane in a line, 

(f) moves the vertical axis, but leaves every point of the horizontal axis where 
it is (a "shear")? 



Inverses 



A matrix is square if it has the same number of rows as columns. If A is a 
square matrix, an inverse is a square matrix B of the same size as A so that 
AB = BA = I. 

3.19 If A, B and C are square matrices, and AB = I and CA = /, prove that 
B = C. In particular, there is only one inverse (if there is one at all). 



So we can unambiguously write the inverse of A (if there is one) as A 1 . 

3.20 If 

check that 

,1 

-1 

3.21 Which lxl matrices have inverses, and what are their inverses? 

3.22 By multiplying out the matrices, prove that any 2x2 matrix 




has inverse 



-c 



ad — be 
as long as ad — bc^ 0. 

3.23 If A and B are invertible matrices, prove that AB is invertible, and 
(ABy 1 = B^A- 1 . 

3.24 Prove that (A^ 1 ) 1 = A, for any invertible square matrix A. 

3.25 If A is invertible, prove that Ax = only for x = 0. 

3.26 If A is invertible, and AB = I, prove that A = B^ 1 and B = A^ 1 . 

Review problems 

3.27 Write down a pair of nonzero 2x2 matrices A and B for which AB = 0. 

3.28 If A is an invertible matrix, prove that Ax = Ay just when x = y. 

3.29 If a matrix M splits up into square blocks like 

M =(o 3 

explain how to find M^ 1 in terms of A^ 1 and D^ 1 . (Warning: for a matrix 
which splits into blocks like 

M -{c 3 

the inverse of M cannot be expressed in any elementary way in terms of the 
blocks and their inverses.) 



Original 




Matrices 



Inverse matrices 





G 




Figure 3.2: Images coming from some matrices, and from their inverses. 



3.30 Figure 3.2 shows how various matrices (on the left hand side) and their 
inverses (on the right hand side) affect vectors. But the two columns are 
scrambled up. Which right hand side picture is produced by the inverse matrix 
of each left hand side picture? 



Elimination by Matrix Multiplication 

Take the 3x3 identity matrix /, and swap the first two rows. Call the resulting 
matrix A: 



A = 



(0 
1 



It turns out that, for any vector x, the vector Ax is just the vector x with the 
first two rows swapped. Why? First, let's see if this is true for the example of 
x = e\. We know that Ae\ is the first column of A. So Ae\ is the first column 



of /, but with the first two rows swapped. The first column of I is e\. So Ae\ 
is ei with the first two rows swapped. The same reasoning exactly works with 
ei replaced by e 2 or e 3 . So if x = e\ or x = e 2 or x = e 3 , then Ax is x with the 
first two rows swapped. Since we can write any vector as x — x\e\ -\-x2e2 -\-x3e3, 
it is enough the check what happens if we take x — e\ and then check x = e 2 
and then check x = e 3 , as we have done. So for any vector x, the vector Ax 
must be just x with the first two rows swapped. A row operation is the process 
of adding a multiple of a row to another row, swapping two row, or rescaling 
a row. The same reasoning works exactly if we start with the n x n identity 
matrix / and let A be the result of carrying out any of the row operations that 
we came across in elimination. Let's make that more precise and summarize 
what we have learned. 

Lemma 3.1. Carry out some row operation on I, or more generally you can 
carry out several row operations to I , as many as you like. Call the resulting 
matrix A. Then for any vector x, the vector Ax is the result of carrying out 
exactly those same row operations on x. 

For example, if we start with the 3x3 identity matrix I, and add 7 row 1 
to row 3 then 

/l 0^ 
A = 1 
\7 

Our lemma claims that Ax is just x with 7 row 1 added to row 3. Let's check: 



Ax = 



Xi 
X2 

K x 3 + 7xi / 

3.31 Which 3x3 matrix S adds —5 row 2 to row 3, and —7 row 1 to row 2? 

3.32 Which 4x4 matrix P takes row 1 to row 2, row 2 to row 3, row 3 to row 
4, and row 4 to row 1? 

Corollary 3.2. Carry carry out several row operations on I , as many as you 
like. Call the resulting matrix A. Then for any matrix B, the matrix AB is 
just the result of applying exactly those same row operations to B. 

Proof. The columns of AB are A times the columns of B. □ 

Corollary 3.3. 7/ A is the matrix you get from I by carrying out some row 
operations, and B is the matrix that you get by carrying out some other row 
operations, then AB is the matrix that you get from I by carrying out first the 
row operations that gave you B, and then those that gave you A. 








' ' X\ 


1 


3 


X2 







\x 3 



Proof. The matrix A acts on a vector x by carrying out those row operations 
that gave us A from /, and the same is true for B. But {AB)x = A(Bx), so 
AB is the matrix that carries out the row operations first of B and then of A. 
So then AB = ABI is the matrix you get by carrying out the row operations 
of B on /, and then the row operations of A. □ 

Corollary 3.4. If A is the matrix that you get from I by carrying out some row 
operations, then the matrix A' 1 is the matrix that you get from I by carrying 
out the inverse row operations, in the reverse order. 



Proof. Let B be the matrix that you get from / by carrying out the inverse row 
operations, in the reverse order. Then AB = I and BA = I, doing and then 
underdoing various operations. □ 



Chapter 4 



Matrices and Row Operations 



We need some practice thinking about examples of matrices. In this chapter, we will 
encounter many different examples of simple types of matrices related to the row 
operations of elimination. 



Permutation Matrices 

A permutation matrix is a matrix obtained by scrambling up the rows of the 
identity matrix. As we just saw, if A is a permutation matrix, and x is a vector, 
then Ax is the result of permuting x by the same scrambling of rows that created 
A in the first place. And as we just saw, the product of any two permutation 
matrices A and B is a permutation matrix C = AB, and C scrambles up the 
rows of a vector x by Cx = A(Bx), i.e. by permuting the rows via B and then 
via A. 

4.1 Prove that a matrix is the permutation matrix of some permutation just 
when 

a. its entries are all O's or l's and 

b. it has exactly one 1 in each column and 

c. it has exactly one 1 in each row. 

4.2 If A is a matrix, the transpose of A is the matrix B — A 1 with the rows 
and columns swapped, so fly = Aji for any i and j. 

a. Use the result of problem 4.1 to prove that if A is a permutation matrix, 
then A 1 is also a permutation matrix. 

b. Prove that for any i and j, Aij = 1 just when Aei = ej. 

c. Prove that A -1 = A 1 , a very fast method to find the inverse of any 
permutation matrix. 

Strictly Lower Triangular Matrices 

A square matrix is strictly lower triangular if it has the form 



/ 1 



1 



S 



\ 



1 / 



31 



with l's on the diagonal, O's above the diagonal, and anything below. 

4.3 Let S be strictly lower triangular. Must it be true that Sij = for i > jl 
What about for j > il 

Lemma 4.1. If S is a strictly lower triangular matrix, and A any matrix, then 
SA is A with SV, (row j) added to row i. In particular, S adds multiples of rows 
to lower rows. 



Proof. For x a vector, 



Sx 



( 1 

5*21 
5*31 

V '■ 



1 

S32 



Xi 

X2 + S21X1 
X3 + S31X1 + S32X2 



V 



/ 



M 

x 2 

X3 

w 



adds S21X1 to x 2 , etc. 

If A is any matrix then the columns of SA are S times columns of A. 



□ 



4.4 Let S be a matrix so that for any matrix A (of appropriate size), SA is A 
with multiples of some rows added to later rows. Prove that S is strictly lower 
triangular. 

4.5 Prove that if R and S are strictly lower triangular then RS is too. 

4.6 Say that a strictly lower triangular matrix is elementary if it has only one 
nonzero entry below the diagonal. Prove that every strictly lower triangular 
matrix is a product of elementary strictly lower triangular matrices. 

Lemma 4.2. Every strictly lower triangular matrix is invertible, and its inverse 
is also strictly lower triangular. 

Proof. Clearly true for 1 x 1 matrices. Let's consider an n x n strictly lower 
triangular matrix S, and assume that we have already proven the result for all 
matrices of smaller size. Write 



where c is a column and A is a smaller strictly lower triangular matrix. Then 



which is strictly lower triangular. 



□ 



A matrix M is strictly upper triangular if it has ones down the diagonal 
zeroes everywhere below the diagonal. 

4.7 For each fact proven above about strictly lower triangular matrices, prove 
an analogue for strictly upper triangular matrices. 

4.8 Draw a picture indicating where some vectors lie in the X1X2 plane, and 
where they get mapped to in the 3/13/2 plane by y = Ax with 



Diagonal Matrices 

A diagonal matrix is one like 



D = 



hi 



V 



*2 



t n J 



(with blanks representing entries). 
4.9 Show by calculation that 



/l 



4 

V 



4 

V 



3\ 
6 



4.10 Prove that a diagonal matrix D is invertible just when none of its diagonal 
entries are zero. Find its inverse. 



Lemma 4.3. If 



D 



\ 



*2 



tn/ 



then DA is A with row 1 scaled by t\, etc. 



Proof. For a vector x, 



ft! 



Dx = 



x 2 



\x n ) 



V 

(tlXi\ 

t 2 x 2 

(just running your fingers along rows and down columns). So D scales row i by 
ti. For any matrix A, the columns of DA are D times columns of A. □ 

Review problems 

4.11 Which diagonal matrix D takes the matrix 



to the matrix 




4.12 Multiply 

0\ 


// 

4.13 Draw a picture indicating where some vectors lie in the x±x 2 plane, and 
where they get mapped to in the yiy 2 plane by y — Ax with each of the following 
matrices playing the part of A: 





Encoding Linear Equations in Matrices 

Linear equations 



x 2 — x 3 = 6 
3x 1 + 7x 2 +4x 3 = 9 
3x i + 5x 2 + 8x 3 = 3 



can be written in matrix form as 



(0 1 -l\ fxA /6\ 

3 7 4 \\x 2 = 9 
13 5 8 / W \3/ 



Any linear equations 



Anxi + A X2 x 2 + ■ 
A 2 \X\ + a 22 x 2 + ■ 

A pl xi + A p2 x 2 + ■ 



■ ■ + A lq x q = bi 

■ ■ + A 2q x q = b 2 

~t~ Ap q x q bp 



become 

/A n A 12 
A 2 \ A 22 

\A V \ A p2 
which we write as Ax = b. 

4.14 Write the linear equations 

xi + 2 x 2 = 7 
3x! + Ax 2 = 8 

in matrices. 



A 



2,1 



f Xl \ 

X2 



ApqJ \Xq/ 



b 2 
\b P J 



Forward Elimination Encoded in Matrix Multiplication 

Forward elimination on a matrix A is carried out by multiplying on the left of 
A by a sequence of permutation matrices and strictly lower triangular matrices. 
For example 





(o 


1 




A = 


3 


7 


4 






5 





Swap rows 1 and 2 (and let's write out the permutation matrix): 



/0 1 0\ /3 7 4\ 

1 \A= 1 -1 
\0 1/ \3 5 8/ 



Add — (row 1) to (row 3): 



1 
1 

,-1 





4 



The string of matrices in front of A just gets longer at each step. Add 2 (row 2) 
to (row 3). 



(l 







1 

\0 2 





/3 




Call this U . This is the echelon form: 

f \ o\ / 1 
U= 1 

\o 2 V V- 1 



/o 
1 



We won't write out these tedious matrices on the left side of A ever again, but it 
is important to see it done once. We will sum up this whole process by writing 
the last line as U — VA, where V is a product of permutation matrices and 
strictly lower triangular matrices. Back substitution is similarly carried out by 
multiplying by strictly upper triangular and invertiblc diagonal matrices. 



Review problems 

4.15 Let P be the 3x3 permutation matrix which swaps rows 1 and 2. What 
does the matrix P" do? Write it down. 

4.16 Let S be the 3x3 strictly lower triangular matrix which adds 2 (row 1) 
to row 3. What does the 3x3 matrix S* 101 do? Write it down. 

4.17 Which 3x3 matrix adds twice the first row to the second row when you 
multiply by it? 

4.18 Which 4x4 matrix swaps the second and fourth rows when you multiply 
by it? 

4.19 Which 4x4 matrix doubles the second and quadruples the third rows 
when you multiply by it? 

4.20 If P is the permutation matrix of a permutation p, what is API 

4.21 If we start with 



A = 



(° 

2 



and end up with 





(2 


3 


4\ 


PA = 





5 


6 




\o 





V 



what permutation matrix is P? 

4.22 If A is a 2 x 2 matrix, and AP = PA for every 2x2 permutation matrix 
P or strictly lower triangular matrix, then prove that A = c I for some number 
c. 

4.23 If the third and fourth columns of a matrix A arc equal, are they still 
equal after we carry out forward elimination? After back substitution? 

4.24 How many pivots can there be in a 3 x 5 matrix in echelon form? 

4.25 Write down the simplest 3x5 matrices you can come up with in echelon 
form and for which 

a. The second and third variables are the only free variables. 

b. There are no free variables. 

c. There are pivots in precisely the columns 3 and 4. 

4.26 Write down the simplest matrices A you can for which the number of 
solutions to Ax = b is 

a. 1 for any b; 

b. for some b, and oo for other b; 

c. for some b, and 1 for other b; 

d. oo for any b. 

4.27 Suppose that A is a square matrix. Prove that all entries of A are positive 
just when, for any nonzero vector x which has no negative entries, the vector 
Ax has only positive entries. 

4.28 Prove that short matrices kill. A matrix is called short if it is wider than 
it is tall. We say that a matrix A kills a vector x if x ^ but Ax = 0. 

Summary 

The many steps of elimination can each be encoded into a matrix multiplication. 
The resulting matrices can all be multiplied together to give the single equation 
U = V A, where A is the matrix we started with, U is the echelon matrix we 
end up with and V is the product of the various matrices that carry out all 
of our elimination steps. There is a big idea at work here: encode a possibly 
huge number of steps into a single algebraic equation (in this case the tiny little 
equation U = VA), turning a large computation into a simple piece of algebra. 
We will use this tiny equation many times. 



Chapter 5 



Finding the Inverse of a Matrix 



Let's use elimination to calculate the inverse of a matrix. 



Finding the Inverse of a Matrix By Elimination 

If Ax = y then multiplying both sides by A -1 gives x = A _1 y, solving for x. 
We can write out Ax — y as linear equations, and solve these equations for x. 
For example, if 

\ -2 N 
-3, 




then writing out Ax = y: 



xi -2 £2 = 2/1 
2xi-3.t 2 = y 2 - 



Let apply Gauss- Jordan elimination, but watch the equations instead of the 
matrices. Add -2 (equation 1) to equation 2. 

x 1 -2x 2 = yi 

x 2 = -2 y\ + 2/2- 

Add 2 (equation 2) to equation 1. 

xi =-3 2/1+22/2 
x 2 = -2yi + 2/2- 

So 

A- x = 

Theorem 5.1. Let A be a square matrix. Suppose that Gauss-Jordan elimina- 
tion applied to the matrix {A I) ends up with (U V) with U and V square 
matrices. A is invertible just when U = I, in which case V = A^ 1 . 

Before the proof, lets have an example. Lets invert 



39 



1 


-2 


1 





2 


-3 








Add -2(row 1) to row 2. 



1 — 1 


-2 


1 








1 


-2 





Make a new pivot 



Add 2 (row 2) to row 1. 



-2 1 

o nn -2 i 



-3 2 

o nn -2 i 



[/ V 

Obviously these are the same steps we used in the example above; the shaded 
part represents coefficients in front of the y vector above. Since U = I, A is 
invertible and 

A- 1 = V 

■(:: :) 

Proof. Gauss- Jordan elimination on (A I) is carried out by multiplying by 
various invertible matrices (strictly lower triangular, permutation, invertible 
diagonal and strictly upper triangular), say like 

(u = M N M N -i . . . M 2 Mi [a 

So 

U = M N M N _! . . . M 2 MiA 
V = M N M N -i . . .M 2 Mi, 

which we summarize as U = VA. Clearly V is a product of invertible matrices, 
so invertible. Thus U is invertible just when A is. 

First suppose that U has pivots all down the diagonal. Every pivot is a 1. 
Entries above and below each pivot are 0, so U = I. Since U — VA, we find 
I = VA. Multiply both sides on the left by V -1 , to see that V^ 1 = A. But 



then multiply on the right by V to see that I = AV. So A and V are inverses 
of one another. 

Next suppose that U doesn't have pivots all down the diagonal. We always 
start Gauss- Jordan elimination on the diagonal, so we fail to place a pivot some- 
where along the diagonal just because we move — > during forward elimination. 
That move makes a pivotless column, hence a free variable for the equation 
Ax = 0. Setting the free variable to a nonzero value produces a nonzero x with 
Ax = 0. By problem 3.25 on page 26, A is not invertible. □ 



Review problems 

5.1 Find the inverse of 



5.2 Find the inverse of 



5.3 Find the inverse of 



5.4 Find the inverse of 
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5.5 Is there a faster method than Gauss- Jordan elimination to find the inverse 
of a permutation matrix? 



I nvertibi lity and Forward Elimination 

Proposition 5.2. A square matrix U in echelon form is invertible just when 
U has pivots all the way down the diagonal, which occurs just when U has no 
zero rows. 



Proof. Applying back substitution to a matrix U which is already in echelon 
form preserves the locations of the pivots, and just rescales them to be 1, killing 
everything above them. So back substitution takes U to I just when U has 
pivots all the way down the diagonal. □ 



(: :) 

(° B 2 \ 

3 
v j 

Theorem 5.3. A square matrix A is invertible just when its echelon form U 
is invertible. 

So we can quickly decide if a matrix is invertible by forward elimination. 
We only need back substitution if we actually need to compute out the inverse. 

Proof. U = VA, and V is invertible, so U is invertible just when A is. □ 

<-(: :) 
-(: 

5.6 Is 



invertible? 
5.7 Is 




invertible? 

5.8 Prove that a square matrix A is invertible just when the only solution x to 
the equation Ax = is x = 0. 



For example, 



is invertible, while 



is not invertible. 



For example, 



has echelon form 



so A is invertible. 



Inversion and Solvability of Linear Equations 



Theorem 5.4. Take a square matrix A. 

a. If the matrix A is invertible then, for any vector b, the equation Ax = b 
has a unique solution x. 

b. If the matrix A is not invertible then the equation Ax = b has either 
no solution or infinitely many, and both of these possibilities occur for 
different choices of b. 

Proof. If A is invertible, then multiplying both sides of Ax — b by A -1 , we see 
that we have to have x = A -1 b. 

On the other hand, suppose that A is not invertible. There is a free variable 
for Ax = b, so no solutions or infinitely many. Lets see that for different choices 
of b both possibilities occur. Carry out forward elimination, say U = V A. Then 
U has a zero row, say row n. We can't solve Ux = e„ (look at row n). So 
set b — V~ l e n and we can't solve Ax = b. But now instead set 6 = and we 
can solve Ax = (for example with x = 0) and therefore solve Ax = with 
infinitely many solutions x, since there is a free variable. □ 



The equations 



asi + 2x2 = 9845039843453455938453 
asi - 2x 2 = 90853809458394034464578 



have a unique solution, because they are Ax = b with 




which has echelon form 




5.9 Suppose that A and B are n x n matrices and AB — I. Prove that A and 
B are both invertible, and that B = A^ 1 and that A = B~ l . 

5.10 Prove that for square matrices A and B of the same size 



(AB) -1 = B- l A~ l 



(and if either side is defined, then the other is and they are equal). 



Review problems 



5.11 Is 




invertible? 



5.12 How many solutions are there to the following equations? 



xt + 2x 2 + 3x 3 = 284905309485083 
x x + 2x 2 + x 3 = 92850234853408 
x 2 + 15x 3 = 4250348503489085. 

5.13 Let A be the n x n matrix which has 1 in every entry on or under the 
diagonal, and in every entry above the diagonal. Find A^ 1 . 

5.14 Let A be the n x n matrix which has 1 in every entry on or above the 
diagonal, and in every entry below the diagonal. Find A -1 . 

5.15 Give an example of a 3 x 3 invcrtiblc matrix A for which A and A* have 
different values for their pivots. 

5.16 Imagine that you start with a 4 x 4 matrix A which might not be invertiblc, 
and carry out forward elimination on (A I). Suppose you arrive at 

* * * *^ 

* * * * 
2 8 3 9' 
15 2/ 

with some pivots somewhere on the first two rows of U . Fact: you can solve 
Ax = b just for those vectors b which solve the equations 

2&i +8b 2 +36 3 +9& 4 = 

b 2 +5b 3 +2b 4 = • 

Explain why. 



(U V) = 
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Chapter 6 



The Determinant 



We can see whether a matrix is invertible by computing a single number, the determi- 
nant. We will learn how to calculate the determinant, and tricks to make it easy to 
find determinants of some types of matrices. 

6.1 Use forward elimination to prove that a 2 x 2 matrix 



matrices, the determinant is complicated. 
Definition 

Determinants are computed as in figure 6.1 on the following page. To compute 
a determinant, run your finger down the first column, writing down plus and 
minus signs in the pattern +,—,+,—,... in front the entry your finger points 
at, and then writing down the determinant of the matrix you get by deleting 
the row and column where your finger lies (always the first column), and add 
up. 

6.2 Prove that 




is invertible just when ad — be =/= 0. 




, the determinant is ad — be. For larger 




Review problems 



6.3 Find the determinant of 




45 




=3 (4 • 2 - 5 • 7) - (2 • 2 - 1 • 7) + 6 (2 • 5 - 1 • 4) . 
Figure 6.1: Computing a 3 X 3 determinant. 



6.4 Find the determinant of 
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6.5 Does A\ Y appear in the expression for det A, when you expand out all of 
the determinants in the expression completely? 

6.6 Prove that the determinant of 

/ 



A = 
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* 



is zero, no matter what number we put in place of the *'s, even if the numbers 
are all different. 

6.7 Give an example of a matrix all of whose entries are positive, even though 
its determinant is zero. 



6.8 What is det/? Justify your answer. 



6.9 Prove that 




= dot A det C, 



for A and C any square matrices, and B any matrix of appropriate size to fit 
in here. 



Easy Determinants 

Lets find the determinant of 



V 



V 



(There are zeros wherever entries are not written.) Running down the first 
column, we only hit the 7. So 

det A = 7 det | 4 | . 



By the same trick: 



det A= (7) (4) det (2) 
= (7)(4)(2). 



Summing up: 

Lemma 6.1. The determinant of a diagonal matrix 



A = 



Pi 



Pi 



\ 



PnJ 



is det A = P1P2 



■Pn- 



We can easily do better: recall that a matrix A is upper triangular if all 
entries below the diagonal are 0. By the same trick again: 



Lemma 6.2. The determinant of an upper triangular square matrix 

fU n t/12 U 13 U 14 ... U ln \ 

U23 U 2 4 ■■■ u 2n 

U33 C/34 ••• u 3n 

u = ' 



t/12 
U22 



V 



is the product of the diagonal terms: det U = U\\Uii . . . U nn . 

Corollary 6.3. A square matrix A is invertible just when det U ^ 0, with U 
obtained from A by forward elimination. 



Proof. The matrix U is upper triangular. The fact that det U ^ says just 
precisely that all diagonal entries of U are not zero, so are pivots — a pivot in 
every column. Apply theorem 5.3 on page 42. □ 



Review problems 



6.10 Find 




6.11 Suppose that U is an invertible upper triangular matrix. 

a. Prove that XJ~ X is upper triangular. 

b. Prove that the diagonal entries of t/~ x are the reciprocals of the diagonal 
entries of U . 

c. How can you calculate by induction the entries of U^ 1 in terms of the 
entries of Ul 



6.12 Let U be any upper triangular matrix with integer entries. Prove that 
U^ 1 has integer entries just when det U = ±1. 



Tricks to Find Determinants 



Lemma 6.4. Swapping any two neighbouring rows of a square matrix changes 
the sign of the determinant. For example, 



det = - det 





2 


V 3 





3 


4 







Proof. It is obvious for 1 x 1 (you can't swap anything) . It is easy to check for a 
2x2. Picture a 3 x 3 matrix A: look at example 6.1 on page 46. For simplicity, 
lets swap rows 1 and 2. Then the plus sign of row 1 and the minus sign of row 
2 are clearly switched in the 1st and 2nd terms in the determinant. In the 3rd 
term, the leading plus sign is not switched. Look at the determinant in the 
3rd term: rows 1 and 2 don't get crossed out, and have been switched, so the 
determinant factor changes sign. So all terms in the determinant formula have 
changed sign, and therefore the determinant has changed sign. The argument 
goes through identically with any size of matrix (by induction) and any two 
neighboring rows, instead of just rows 1 and 2. □ 



Lemma 6.5. Swapping any two rows of a square matrix changes the sign 
of the determinant, so det PA = — det A for P the permutation matrix of a 
transposition. 



Proof. Suppose that we want to swap two rows, not neighboring. For concrete- 
ness, imagine rows 1 and 4. Swapping the first with the second, then second 
with third, etc., a total of 3 swaps will drive row 1 into place in row 4, and 
drives the old row 4 into row 3. Two more swaps (of row 3 with row 2, row 2 
with row 1) puts everything where we want it. 

More generally, to swap two rows, start by swapping the higher of the two 
with the row immediately under it, repeatedly until it fits into place. Some 
number s of swaps will do the trick. Now the row which was the lower of the 
two has become the higher of the two, and we have to swap it s — 1 swaps into 
place. So 2s — 1 swaps in all, an odd number. □ 

6.13 If a square matrix has two rows the same, prove that it has determinant 
0. 

6.14 Find 2x2 matrices A and B for which 

det(^4 + B)^dctA + det B. 

So det doesn't behave well under adding matrices. But it does behave well 
under adding rows of matrices. 
Watch each row: 

det ( 1 + 5 ' + 6 | = det | * ~ | + det 

Theorem 6.6. The determinant of any square matrix scales when you scale 
across any row like 

dot ( ) :7.,l,l' 
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3 4 J ^34 

or when you scale down any column like 

7-1 2 

It adds when you add across any row like 

det^ + 5 2 + 6 )=det^ "l+det 

or when you add down any column like 
2 + 5 N 






Proof. To compute a determinant, you pick an entry from the first column, and 
then delete its row and column. You then multiply it by the determinant of 
what is left over, which is computed by picking out an entry from the second 
column, not from the same row, etc. If we ignore for a moment the plus and 
minus signs, we can see the pattern emerging: you just pick something from 
the first column, and cross out its row and column, 

/ \ 



V / 

and then something from the second column, and cross out its row and column, 



\ 



V / 

and so on: 



V / 

Finally, you have picked one entry from each column, all from different rows. In 
this example, we picked A31, A52, A23, An, Multiply these together, and 
you get just one term from the determinant: ^31^452^23^14^45. Your term 
has exactly one entry from the first column, and then you crossed out the first 
column and moved on. Suppose that you double all of the entries in the first 
column. Your term contains exactly one entry from that column, A31 in our 
example, so your term doubles. Adding up the terms, the determinant doubles. 

In the same way, scaling any column, you scale your entry from that column, 
so you scale your term. You scale all of the terms, so you scale the determinant. 

When you cobbled together your term, you picked out an entry from some 
row, and then crossed out that row. So you didn't use the same row twice. 
There are as many rows as columns, and you picked an entry in each column, 
so you have picked as many entries as there are rows, never using the same row 
twice. So you must have picked out exactly one entry from each row. In our 
example term above, we see this clearly: the rows used were 3, 5, 2, 1,4. By the 
same argument as for columns, if you scale row 2, you must scale the entry A23, 
any only that entry, so you scale the term. Adding up all possible terms, you 
scale the determinant. 



Lets see why we can add across rows. If I try to add entries across the first 
row, a single term looks like 





= (4+9)(...) 

where the (...) indicates all of the other factors from the lower rows, which we 
will leave unspecified, 




since we keep all of the entries in the lower rows exactly the same in each matrix. 
This shows that each term adds when you add across a single row, so the sum 
of the terms, the determinant, must add. This reasoning works for any size of 
matrix in the same way. Moreover, it works for columns just in the same way 
as for rows. □ 



6.15 What happens to the determinant if I double the first row and then triple 
the second row? 

The determinant is the sum over all choices you could make of rows to pick 
at each step; and of course, there are some plus and minus signs which we are 
still ignoring. 

6.16 Draw pictures like those in the proof above of patterns of crossing out 
rows and columns, and explain which term each one computes, for determinants 
of 

a. 2x2, 

b. 3x3, and 

c. 4 x 4 
matrices. 

Proposition 6.7. Suppose that S is the strictly upper or strictly lower trian- 
gular matrix which adds a multiple of one row to another row. Then 

det SA = det A. 

i.e. we can add a multiple of any row to any other row without affecting the 
determinant. 



Proof. We can always swap rows as needed, to get the rows involved to be the 
first and second rows. Then swap back again. This just changes signs somehow, 
and then changes them back again. So we need only work with the first and 
second rows. For simplicity, picture a 3 x 3 matrix as 3 rows: 





( al \ 




0,2 




K «3 ) 



Adding s (row 1) to (row 2) gives 



which has determinant 



det 



V 



ax 

a 2 + sai 

"3 



V 



= det 




/ 




+ s det 



ai 
ai 



by the last lemma. The second determinant vanishes because it has two identical 
rows. The general case is just the same with more notation: we stuff more rows 
around the three rows we had above. □ 



6.17 Which property of the determinant is illustrated in each of these examples? 
(a) 



(b) 



(c) 



flO 



det 



det 



det 
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-3 





(l 





5 det 



= - det 



det 
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-1 
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-3 
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Chapter 7 



The Determinant via Elimination 



The fast way to compute the determinant of a large matrix is via elimination. 

The fast formula for the determinant 
Theorem 7.1. Via forward elimination, 

I ± (product of the pivots) if there is a pivot in each column, 



where 

{+ if we make an even number of row swaps during forward elimination, 
— otherwise. 

In particular, A is invertible just when det A =/= 0. 
Forward elimination takes 



with one row swap so det A = — (2)(7) = —14. 

The fast formula isn't actually any faster for small matrices, so for a 2 x 2 
or 3 x 3 you wouldn't use it. But we need the fast formula anyway; each of the 
two formulas gives different insight. 

Proof. We can see how the determinant changes during elimination: adding 
multiples of rows to other rows does nothing, swapping rows changes sign. □ 

7.1 Use the fast formula to find the determinant of 




otherwise. 





5 



2 5 7 
\2 6 11 



53 



7.2 Just by looking, find 



det 



/1001 


1002 


1003 


1004\ 


2002 


2004 


2006 


2008 


2343 


6787 


1938 


4509 


\9873 


7435 


2938 


9038/ 



7.3 Prove that a square matrix is invertible just when its determinant is not 
zero. 



Review problems 

7.4 Find the determinant of 
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7.5 Find the determinant of 
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7.6 Find the determinant of 



7.7 Find the determinant of 



7.8 Find the determinant of 



7.9 Find the determinant of 
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7.10 Prove that a square matrix with a zero row has determinant 0. 



7.11 Prove that det PA = (— l) Ar det A if P is the permutation matrix of a 
product of N transpositions. 

7.12 Use the fast formula to find the determinant of 

/o 



A = 



3 



7.13 Prove that the determinant of any lower triangular square matrix 

\ 



L 



/ill 






L21 


L22 




L31 


L32 




L41 


L42 


L43 


\L n i 


L n 2 


L n 3 



L, 



(n-l) 



(with zeroes above the diagonal) is the product of the diagonal terms: det L 
L11L22 ■ ■ ■ inn- 



Determinants Multiply 

Theorem 7.2. det (AB) = det(A) det(B), for any square matrices A and B 
of the same size. 

Proof. Suppose that detv4 = 0. By the fast formula, A is not invertible. Prob- 
lem 5.10 on page 43 tells us that therefore AB is not invertible, and both 
det(AB) and det(A) det(B) are 0. So we can safely suppose that det A ^ 0. 
Via Gauss- Jordan elimination, any invertible matrix is a product of matrices 
each of which adds a multiple of one row to another, or scales a row, or swaps 
two rows. Write A as a product of such matrices, and peel off one factor at a 
time, applying lemma 6.4 on page 48 and proposition 6.7 on page 51. □ 

If 

/l 4 6\ 1 1 0\ 

A= 2 5 ,-8= 2 2 , 

\0 3/ \7 5 4/ 

then it is hard to compute out AB, and then compute out det ^45. But 
det AB = det A det B = (1)(2)(3)(1)(2)(4) =48. 



Transpose 



The transpose of a matrix A is the matrix A whose entries are A*- = Aj 
(switching rows with columns). Flip over the diagonal: 



10 3 5 
2 40 6, 
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A = 
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40 




V 5 


6 



7.14 Find the transpose of 

h 2 

A= 4 5 

\0 

7.15 Prove that 

(AS)' = 

(The transpose of the product is the product of the transposes, in the reverse 
order.) 

7.16 Prove that the transpose of any permutation matrix is a permutation 
matrix. How is the permutation of the transpose related to the original permu- 
tation? 

Corollary 7.3. 

det A = det A* 



Proof. Forward elimination gives U = V A, U upper triangular and V a product 
of permutation and strictly lower triangular matrices. Tranpose: IP — A t V t . 
But V 1 is a product of permutation and strictly upper triangular matrices, with 
the same number of row swaps as V, so det V 1 = det V = ±1. The matrix IP 
is lower triangular, so det IP is the product of the diagonal entries of J7* (by 
problem 7.13 on the preceding page), which are the diagonal entries of U, so 
det IP = det U. □ 

Expanding Down Any Column or Across Any Row 

Consider the "checkerboard pattern" 

+ - + -••• 
- + - + 



Theorem 7.4. We can compute the determinant of any square matrix A by 
picking any column (or any row) of A, writing down plus and minus signs from 
the same column (or row) of the checkboard pattern matrix, writing down the 



entries of A from that column (or row), multiplying each of these entries by 
the determinant obtained from deleting the row and column of that entry, and 
adding all of these up. 



For 

h 2 

A = 1 4 
\6 7 

if we expand along the second row, we get 



dct A = - (l)det 



+ (4) det 
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Proof. By swapping columns (or rows), we change signs of the determinant. 
Swap columns (or rows) to get the required column (or row) to slide over to 
become the first column (or row). Take the sign changes into account with the 
chcckboard pattern: changing all plus and minus signs for each swap. □ 

7.17 Use this to calculate the determinant of 



/ 1 
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-1702 
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493/ 



Summary 

Determinants 

(a) scale when you scale across a row (or down a column), 

(b) add when you add across a row (or down a column), 

(c) switch sign when you swap two rows, (or when you swap two columns), 

(d) don't change when you add a multiple of one row to another row (or a 
multiple of one column to another column), 



(e) don't change when you transpose, 

(f) multiply when you multiply matrices. 
The determinant of 

(a) an upper (or lower) triangular matrix is the product of the diagonal entries. 

(b) a permutation matrix is (— 1) # of trans P° sitions . 

(c) a matrix is not zero just when the matrix is invertible. 

(d) any matrix is det A = (—1) N det U, if A is taken by forward elimination 
with iV row swaps to a matrix U. 

7.18 If A is a square matrix, prove that 

det (A k ) = (detA) fc 

for k= 1,2,3,.... 

7.19 Use this last exercise to find 

j et ^2222444466668888^ 

where 

A-(» ' V 

\l 1234567890 J 

7.20 If A is invertible, prove that 

det (A- 1 ) = -JL. 
v ' det A 

Review problems 

7.21 What are all of the different ways you know to calculate determinants? 

7.22 How many solutions are there to the following equations? 

xi + 1010x2 + 130923x 3 = 2839040283 
2x 2 + 23932x 3 = 2390843248 
3x 3 = 98234092384 

7.23 Prove that no matter which entry of an n x n matrix you pick (n > 1), 
you can find some invertible n x n matrix for which that entry is zero. 



Bases and Subspaces 



Chapter 8 
Span 



We want to think not only about vectors, but also about lines and planes. We will 
find a convenient language in which to describe lines and planes and similar objects. 

The Problem 

Look at a very simple linear equation: 



There are many solutions. Each is a point in R 3 , and together they draw out a 
plane. But how do we write down this plane? The picture is useless — we can't 
see for sure which vectors live on it. We need a clear method to write down 
planes, lines, and similar things, so that we can communicate about them (e.g. 
over the telephone or to a computer). 

One method to describe a plane is to write down an equation, like x\ + 
2.T2 + X3 — 0, cutting out the plane. But there is another method, which we 
will often prefer, building up the plane out of vectors. 



Span 

Consider the equations 




Figure 8.1: The so- 
lutions of an equation 
forming a plane. 



X\ + 2x 2 + #3 = 0. 



(8.1) 



X\ + 2X2 



-7x4 = 
X3 + -£4 = 



Solutions have 



x\ = —2 X2 + 7 X4 



x 3 = -x 4 , 



Gl 



giving 



M 

X2 

\xj 

X2 
— X4 
•''4 
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X2 



X4 




-1 

v i y 



But £2 and x^ are free — they can be anything. The solutions are just arbitrary 
"combinations" of 







(?\ 


i 


and 








-1 









We can just remember these two vectors, to describe all of the solutions. 

A multiple of a vector v is a vector cv where c is a number. A linear 
combination of some vectors V\, V2, ■ ■ ■ ,v v in W 1 is a vector 



C1V1 + C2V2 + 



for some numbers c% , Ox , . . . , c p (a sum of multiples) . The span of some vectors 
is the collection of all of their linear combinations. 



The Solution 

We can describe the plane of solutions of equation 8.1 on the previous page: it 
is the span of the vectors 









: 




; 









That isn't obvious. (You can apply forward elimination to check that it is 
correct.) But immediately we see our next problem: you might describe it as 
this span, and I might describe it as the span of 









: 




• 









How do we see that these are the same thing? 

How to Tell if a Vector Lies in a Span 

If we have some vectors, lets say 





(l\ 




(l\ 


x 1 = 


2 


, x 2 = 







W 




v-V 



how do we tell if another vector lies in their span? Lets ask if 

y = 



lies in the span of X\ and x 2 . So we are asking if y is a linear combination 
C\X\ + c 2 x 2 . 

Solving the linear equations 

ci + c 2 = 1 
2ci =4 
3ci - c 2 = 7 

just means finding numbers c\ and c 2 for which 




writing y as a linear combination of x\ and X2. Solving linear equations is 
exactly the same problem as asking whether one vector is a linear combination 
of some other vectors. 

8.1 Write down some linear equations, so that solving them is the same problem 
as asking whether 

" 1 


>-2y 

is a linear combination of 




A pivot column of a matrix A is a column in which a pivot appears when 
we forward eliminate A. 



The matrix 



has echelon form 



.4 : 







U 





1 

1 





so columns 1 and 3 of A: 



( 







I— 1 
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A = 

are pivot columns. 

Lemma 8.1. Write some vectors into the columns of a matrix, say 
A = 



X-2 



II 



and apply forward elimination. Then y lies in the span of X\, X2, ■ ■ ■ , x p just 
when y is not a pivot column. 

Proof. As in our example on page on the preceding page, the problem is precisely 
whether we can solve the linear equations whose matrix is A, with y the column 
of constants. We already know that linear equations have solutions just when 
the column of constants is not a pivot column. □ 

Applied to our example, this gives 

A = (x-i x 2 yj 



(l 

2 

\3 

to which we apply forward elimination: 
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1) to row 3 
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Add — 2(row 2) to row 3. 
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There is no pivot in the last column, so y is a linear combination of x\ and x%, 
i.e. lies in their span. (In fact, in the echelon form, we see that the last column 
is twice the first column minus the second column. So this must hold in the 
original matrix too: y — 2x\ — x%-) 



8.2 What if we have a lot of vectors y to test? Prove that vectors yi,y2, 
all lie in the span of vectors x\, x-i, . . . , x p just when the matrix 



. Vq 



\X\ X2 ••• a 
has no pivots in the last q columns. 



!J2 



Vq 



Review problems 

8.3 Is the span of the vectors 




the same as the span of the vectors 

/o 
V 2 

8.4 Describe the span of the vectors 




8.5 Does the vector 



lie in the span of the vectors 





8.6 Does the vector 



lie in the span of the vectors 
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8.7 Docs the vector 



lie in the span of the vectors 



8.8 Docs the vector 



lie in the span of the vectors 
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8.9 Find a linear equation satisfied on the span of the vectors 
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Subspaces 

Picture a straight line through the origin, or a plane through the origin, 
generalize this picture: a subspace P of R" is a collection of vectors in R 
that 

a. P is not empty (i.e. some vector belongs to the collection P) 

b. If x belongs to P, then ax does too, for any number a. 



c. If x and y belong to P, then x + y does too. 
We can see in pictures that a plane through the origin is a subspace: 







My plane is not 
empty: the origin 
lies in my plane 



Scale a vector 
from my plane: 
it stays in that 
plane 



Add vectors from 
my plane: the 
sum also lies in 
my plane 



8.10 Prove that belongs to every subspace. 



8.11 Prove that if a subspace contains some vectors, then it contains their 
span. 

Intuitively, a subspace is a flat object, like a line or a plane, passing through 
the origin of W l . The set P of vectors 

--(:) 

for which x\ + 2x2 = is a subspace of M 2 , because 

a. x — satisfies x\ + 2x-2 = (so P is not empty). 

b. If x satisfies x\ + 2x2 = 0, then ax satisfies 

(ax) i + 2(ax)2 = ax\ + 2aX2 
= a (xi + 2 x 2 ) 
= 0. 

c. If x and y are points of P, satisfying 

x\ + 2x2 — 
Vl + 2y 2 = 



then x + y satisfies 



(xi +yi) + 2 (x 2 + y 2 ) = (xi + 2x 2 ) + (yi + 2y 2 ) 
= 0. 



8.12 Is the set S of all points 



Xi 
^2, 



of the plane with i2 = la subspace? 



8.13 Is the set P of all points 




x 



\X3 



with x\ + X2 + x 3 = a subspace? 

The word "subspace" really means just the same as "the span of some 
vectors," as we will eventually see. 

Proposition 8.2. The span of a set of vectors is a subspace; in fact, it is the 
smallest subspace containing those vectors. Conversely, every subspace is a 
span: the span of all of the vectors inside it. 

In order to make this proposition true, we have to change our definitions 
just a little: if we have an empty collection of vectors (i.e. we don't have any 
vectors at all), then we will declare that the span of that empty collection is 
the origin. 

If we have an infinite collection of vectors, then their span just means the 
collection of all linear combinations we can build up from all possible choices 
we can make of any finite number of vectors from our collection. We don't 
allow infinite sums. We would really like to avoid using spans of infinite sets of 
vectors; we will address this problem in chapter 9. 

Proof. Given any set of vectors X in R n , let U be their span. So any vector in 
U is a linear combination of vectors from X. Scaling any linear combination 
yields another linear combination, and adding two linear combinations yields 
a further linear combination, so U is a subspace. If W is any other subspace 
containing X, then we can add and scale vectors from W, yielding more vectors 
from W, so we can make linear combinations of any vectors from W making 
more vectors from W. Therefore W contains the span of X, i.e. contains U. 

Finally, if V is any subspace, then we can add and scale vectors from V to 
make more vectors from V, so V is the span of all vectors in V. □ 

8.14 Prove that every subspace is the span of the vectors that it contains. 
(Warning: this fact isn't very helpful, because any subspace will either contain 
only the origin, or contain infinitely many vectors. We would really rather only 
think about spans of finitely many vectors. So we will have to reconsider this 
problem later.) 

8.15 What are the subspaces of R? 

8.16 If U and V are subspaces of R": 

a. Let W be the set of vectors which either belong to U or belong to V. Is 
W a subspace? 

b. Let Z be the set of vectors which belong to U and to V. Is Z a subspace? 



Review problems 

8.17 Is the set X of all points 



x = 



of the plane with X2 = x\ a subspace? 

8.18 Which of the following are subspaces of M 4 ? 

a. The set of points x for which x\x 4 = x 2 xz- 

b. The set of points x for which 2x i — 3x 2 . 

c. The set of points x for which x\ + x 2 + x% + x 4 = 0. 

d. The set of points x for which x\, x 2 , x% and X4 are all > 0. 

8.19 Is a circle in the plane a subspace? Prove your answer. Draw pictures to 
explain your answer. 

8.20 Which lines in the plane are subspaces? Draw pictures to explain your 
answer. 



Summary 

We have solved the problem of this chapter: to describe a subspace. You write 
down a set of vectors spanning it. If I write down a different set of vectors, 
you can check to see if mine are linear combinations of yours, and if yours are 
linear combinations of mine, so you know when yours and mine span the same 
subspace. 



Chapter 9 



Bases 



Our goal in this book is to greatly simplify equations in many variables by changing 
to new variables. In linear algebra, the concept of changing variables is replaced with 
the more concrete concept of a basis. 



Definition 

A basis is a list of "just enough" vectors to span a subspace. For example, we 
should be able to span a line by writing down just one vector lying in it, a plane 
with just two vectors, etc. 

A linear relation among some vectors x\, X2, ■ ■ ■ ,x p in WL n is an equation 



where C\, Cq,, . . . , c p are not all zero. 

A set of vectors is linearly independent if the vectors admit no linear relation. 
A set of vectors is a basis of R" if (1) the vectors are linearly independent and 
(2) adding any other vector into the set would render them no longer linearly 
independent . 

The vectors 



satisfy the linear relation 2x\ — x-i — 0. 
Properties 

Lemma 9.1. The columns of a matrix are linearly independent just when each 
one is a pivot column. 



r 

Figure 9.1: Only this 
plane contains and 
these two vectors . 
Three- legged tables 
don't wobble, unless all 
of the feet of the table 
legs lie on the same 
straight line. 




Figure 9.2: The vector 
sticking up is linearly in- 
dependent of the other 
two vectors. 




Proof. Obvious from lemma 8.1 on page 64. 



□ 



9.1 Is 




a basis of M 2 ? 
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9.2 The standard basis of R™ is the basis ei, e2, ■ • ■ , e„ (where ei is the first 
column of /„, etc.). Prove that the standard basis of R™ is a basis. 

9.3 Prove that there is a linear relation between some vectors w\, 11)2, . . . ,w q 
just when one of those vectors, say Wk, is a linear combination of earlier vectors 

Wi,W 2 , ■ ■ .,Wk-l- 

Theorem 9.2. Every linearly independent set of vectors in R™ consists in at 
most n vectors, and consists in exactly n vectors just when it is a basis. 

Proof. Suppose that x\, x%, . . . , x p are a linearly independent. Let 



There is either one pivot or no pivot in each row. So the number of rows is at 
least as large as the number of pivots. There are n rows. There is one pivot in 
each column, so p pivots. So p < n. 

If p = n, we have one pivot in each row, so adding another vector (another 
column) can't add another pivot. Therefore adding any other vector to the 
vectors x\, X2, ■ ■ ■ , x p would break linear independence. 

If p < n, then we have zero rows after forward elimination. Suppose that 
forward elimination yields U — VA. Then (U e p +i) has more pivot columns 
than U has, so (A V~ 1 e p+ i) has more pivot columns than A has. Thus adding 
a new vector x p+ \ = V~ 1 e p+ i to the collection of vectors Xi,X2, ■ ■ ■ ,x p , we 
have a larger linearly independent collection. □ 

9.4 Prove that every linearly independent set of vectors in K n belongs to a 
basis. 

Lemma 9.3. A set of vectors iti , U2, ■ ■ ■ , u n is a basis of M. n just when every 
vector b in R" can be written as a linear combination 




x 




b = ami + 02^2 + • ■ ■ + a n u. 



for a unique choice of numbers a%, 02, . . . , a. 



Proof. Let 





a 



\a n J 



and apply theorem 5.4 on page 43 to the equation Aa = b. 



□ 



Review problems 



9.5 Are the vectors 

mm 

linearly independent? 

9.6 Are the vectors 




a basis? 

9.7 Can you find matrices A and B so that A is 3 x 5 and B is 5 x 3, and 
AB = 1? 

9.8 Suppose that A is 3 x 5 and B is 5 x 3, and that AB is invertible. Must 
the columns of B be linearly independent? the rows of Bl the columns of A? 
the rows of A? 

9.9 Give an example of a 3 x 3 matrix for which any two columns are linearly 
independent, but the three columns together are not linearly independent. Can 
such a matrix be invertible? 



The Change of Basis Matrix 

The change of basis matrix F associated to a basis U\ 1 u 2l ■ ■ ■ , u n of K™ is the 
matrix 

F = (ui u 2 ... m„) ■ 

Note that Fe\ — ui,Fe2 — u 2 , ■ ■ ■ ,Fe n — u n . So taking x to Fx is a 
"change of basis", taking the standard basis to the new basis. 

9.10 Prove that an n x n matrix C is the change of basis matrix of a basis 
just when the equation Cx = has x = as its only solution, which occurs just 
when C is invertible. 

Suppose that you and I look at the sky and watch a falling star. You measure 
its position against the fixed choice of basis ei, e 2l e^, while I measure against 
some funny choice of basis ui,u 2 ,u^. 

The actual position is some vector p in K 3 . Lets say 

p = x\e\ + x 2 e 2 + £363 as you measure it, 
= y\U\ + y 2 u 2 + J/3M3 as I measure it. 

Let 

F = \ ux u 2 u 3 J 



be the change of basis matrix, so that Fe\ — m, etc. So F takes your basis to 
mine. If we let 









X 2 


and y = 




W 




w 



then in your basis: 

p = x\e\ + x 2 e 2 + x 3 e 3 
= x 

but in mine: 



P = 2/1 Ml + V2U 2 + V3.U3 
= V\Fe-i + y 2 Fe 2 + y 3 Fe 3 
= F(y 1 e 1 +y 2 e 2 + y 3 e 3 ) 
= Fy. 

So x — Fy converts my measurements to yours. 

Suppose that we change variables by x = Fy and so y = F~ 1 x, with F 
some invcrtible matrix. Then any matrix A acting on the x variables by taking 
x to Ax is represented in y variables as 

F- 1 A F 



turn x's to y's act on x's 
the matrix F~ 1 AF. 



turn y's to x's 



9.11 Take 



F = 




Compute F _1 AF. 

9.12 A shower of falling stars fall to Earth. Each star falls from a position 



to a position on the ground 



Ax 



What is the matrix A? Suppose that I measure the positions of the stars against 
the basis 




Find the change of basis matrix F, and find F 1 AF, the matrix that describes 
how each star falls from the sky as measured against my basis. 



Review problems 
9.13 Is 




a basis of M 3 ? 

9.14 If A is a matrix, show how each vector which A kills determines a linear 
relation between the columns of A, and vice versa. 



9.15 Are 

CM:) 

linearly independent? 

9.16 Write down a basis of M 2 other than the standard basis, and prove that 
your basis really is a basis. 



9.17 Is 




a basis of M 2 ? 



9.18 Is 




a change of basis matrix? If so, for what basis? 



9.19 If x\, X2, ■ ■ ■ , x n and y\, y2, ■ ■ ■ , y n are two bases of R™, prove that there 
is a unique invertible matrix A so that Ax\ = y\,Ax2 = yi, etc. 



Bases of Subspaces 



We can write down a subspace, by writing down a spanning set of vectors. But 
you might write down more vectors than you need to. We want to squeeze 
the description down to the bare simplest minimum, throwing out redundant 
information. 

If V is a subspace of M. n , a basis of V is a set of linearly independent vectors 
from V, so that adding any other vector from V into the set would render them 
no longer linearly independent. 

The vectors 



Lemma 9.4. If some vectors span a subspace, then putting them into the 
columns of a matrix, the pivot columns form a basis of the subspace. 

Lets find a basis for the span of the vectors 




are a basis for the subspace V in M 3 of vectors of the form 




Obviously: 




Put them into a matrix 



V 



1 



o 



i 



i 











1 




Forward eliminate: 








1 



1 




so the first and second columns are pivot columns. Therefore 




are a basis for the span. 



Proposition 9.5. Every subspace of K." has a basis. Moreover, any basis 
Vi, v^, • ■ ■ , v p of a subspace V o/K™ lives in a basis v%, Ua, • • • , v p , Wi, W2, ■ ■ ■ , w q 
ofR n . 

Proof. If V only contains the vector, then we can take no vectors as a basis 
for V, and let W\, W2, . . . ,w n be any basis for R n . On the other hand, if V 
contains a nonzero vector, then pick as many linearly independent vectors from 
V as possible. By theorem 9.2 on page 72, we could only pick at most n vectors. 
They must span V, because otherwise we could pick another one. If V = M", 
then we are finished. Otherwise, pick as many vectors from R n as possible 
which are linearly independent of v\, t>2, . . . , v p . Clearly we stop just when we 
hit a total of n vectors. □ 

Dimension 

Do all bases look pretty much the same? 

Theorem 9.6. Any two bases of a subspace have the same number of vectors. 

Proof. Imagine two bases, say x\, x%, . . . ,x p and y\, j/2> • • • > Vq-, for the same 
subspace. Forward eliminate 



( xi 



x 2 



x P yi y2 



yielding 



V 



Each x vector generates a pivot, p pivots in all, straight down the diagonal. 
Forward eliminate the right hand portion of the matrix, yielding 



\ 



V / 



giving at most p pivots because of the zero rows. Each y vector generates a 
pivot. So there aren't more than p of these y vectors. Thus no more y vectors 
than x vectors. Reversing the roles of x and y vectors, we find that there can't 
be more x vectors than y vectors. □ 

9.20 Prove that every subspace of W 1 has a basis with at most n vectors. 

The dimension of a subspace is the number of vectors in any basis. Write 
the dimension of a subspace U as dim U. 

Review problems 

9.21 Consider the vectors in M. n of the form — ej (for all possible values of i 
and j from 1 to n). Find a basis for the subspace they span. 

Summary 

• A subspace is a flat thing passing through 0. 

• A basis for a subspace is a collection of just enough vectors to span the 
subspace. 

• A change of basis matrix is a basis organized into the columns of a matrix. 

Uniqueness of Reduced Echelon Form 

When we carry out elimination, we choose rows to swap. 

9.22 Find the simplest matrix A you can with two different ways of carrying 
out forward elimination, with different results. 

Recall that Gauss- Jordan elimination means forward elimination followed 
by back substitution. The matrix resulting from Gauss- Jordan elimination is 
said to be in reduced echelon form. 

Theorem 9.7. The result of Gauss-Jordan elimination does not depend on the 
choices made of which rows to swap. 

Proof. Suppose that U and W are two different eliminations of the same matrix 
A, obtained using different choices of rows to swap. The first pivot column is 
just the first nonzero column of A. The second pivot column is the earliest 
column which is linearly independent of the first pivot column, etc. This is 
true for A, and doesn't change under forward elimination or back substitution. 
Therefore A and U and W have the same pivot columns. After elimination, the 
first pivot column becomes ei, the second becomes e 2 , etc. So all of the pivot 
columns of U and W must be identical. 

Every pivotless column is a linear combination of earlier pivot columns, and 
the coefficients in this linear combination are not affected by Gauss- Jordan 



elimination. Therefore the pivotless columns of U and W are the same linear 
combinations of the pivot columns. The pivot columns are the same, so all 
columns are the same. □ 



9.23 The rank is the number of pivots in the forward elimination. Prove that 
the rank of a matrix does not depend on which rows you choose when forward 
eliminating. 



Chapter 10 



Kernel and Image 



Each matrix A has two important subspaces associated to it: its kernel (the vectors it 
kills), and its image (the vectors b for which you can solve Ax = b). 



If A is any matrix, say px q, then the vectors x in R 9 for which Ax = (vectors 
"killed" by A) form a subspace of M. q called the kernel of A, and written ker A 
The kernel is a subspace, because 

a. belongs to the kernel of any matrix A, since AO = (everything kills 0). 

b. If Ax = and Ay = 0, then A(x + y) = Ax + Ay — (when you kill two 
vectors, you kill their sum). 

c. If Ax = 0, then A (ax) — aAx — (when you kill a vector, you kill its 
multiples) . 

10.1 If a matrix is wider than it is tall (a "short" matrix), then its kernel 
contains nonzero vectors. 

10.2 Prove that the kernel of AB contains the kernel of B. Does it have to 
contain the kernel of Al 

We will often need to find kernels of matrices. To rapidly calculate the kernel 
of a matrix, for example 



Kernel 



.4 




a. Carry out forward elimination and back substitution. 




l 

2 
3 
2 




1 
\ 







b. Cut out all zero rows. 








1 



l 

2 
3 
2 
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c. Change the signs of all entries after each pivot. 




1 



5x3 



\x± 



= to 



(This corresponds to changing equations like x\ 

X\ = — I x 3 — I X4. Think of it as moving everything after the pivot over 
to the right hand side, although we won't actually move anything.) 
Stuff in whatever rows from the identity matrix you need into your matrix, 
so that it ends up with nonzero entries all down the diagonal. 



/l 



Vo 





1 / 



We won't mark the new rows with pivots. Each new row corresponds to 
setting one of the free variables to 1 and the others to 0. 
Cut out all of the pivot columns. The remaining columns are a basis for 
the kernel. 







H\ 


3 
2 




1 

2 


1 


1 





V J 




V 1 / 



10.3 Apply this algorithm to the matrix 

/o 

2 



.4 



V-2 



and check that Ax = for each vector x in your resulting basis for the kernel. 
Lemma 10.1. The algorithm works, giving a basis for the kernel of any matrix. 

Proof. Each vector in the kernel is obtained by setting arbitrary values for 
the free variables, and letting the pivots solve for the other variables. Let v\ 
be the vector in the kernel which has value 1 for the 1 st free variable, and 
for all other free variables. Similarly, make a vector V2, ^3, • • • , v s for each free 
variable-suppose that there are s free variables. The kernel is a subspace, so 
each linear combination 



C1V1 + c 2 v 2 H h c s v s 



lies in the kernel. This linear combination has value c\ for the first free variable, 
C2 for the second, etc. (just looking at the rows of the free variables). Each 



vector in the kernel has some values Ci , C2, • • . , c s for the free variables. So each 
vector in the kernel is a unique linear combination of v%, V2, • ■ ■ , v g . Suppose that 
we find a linear relation among v\, v%, . . . , v s , say c\V\ + C2V2 + • • • + c s v s = 0. 
Look at the row in which v\ has a 1 and all of the other vectors have 0's: the 
linear relation gives ci = in that row. Similarly all of ci,c%, . . . ,c a must 
vanish, so there is no linear relation among these vectors. Therefore the vectors 
v\ , V2 , • ■ • , v s form a basis for the kernel. 

Finally, we need to see why these vectors Vi, V2, ■ ■ ■ , v a are precisely the 
vectors which come out of our process above. First, look at our example. The 
reduced echelon form turns back into equations as 



•1'2 



1^3 
4x 3 



Solving for pivots means subtracting off: 



X! 



X 2 



■\xi 

■k X4 



'\x 3 

§Z3 



2 X4 
2 Xi - 



All free variables line up on the right hand side, and we have changed the signs 
of their coefficients. Setting x% — 1 and £4 = 0, go down the right hand side, 
killing the X4 entries, and putting x% — 1 in each x 3 entry, i.e. writing down 
just the entries from the £3 column: 



Vi = 



V y 



The general algorithm works in the same way: if we put all free variables on to 
the right hand side, and then set one free variable to 1 ("turn it on") and the 
others to 0's ("turn them off"), we can picture this as "turning on" the column 
associated to that free variable. Each pivot solves for a pivot variable — the 
value of that pivot variable is the entry in the corresponding row of the "turned 
on" column. □ 



10.4 Give an example of a square matrix whose kernel is not the kernel of its 
transpose. 

10.5 Draw a picture of the kernel for each of 

Corollary 10.2. The dimension of the kernel of a matrix is the number of 
pivotless columns after forward elimination. 

Another way to say it: the dimension of the kernel of a matrix A is the number 
of free variables in the equation Ax = 0. 



Review problems 



10.6 Find a basis for the kernel of 



(\ 











o\ 





1 








1 








1 


2 


2 


\o 











0/ 



10.7 Find a basis for the kernel of 





1 








o\ 








1 

















1 





\o 











1/ 



10.8 Find a basis for the kernel of 



/ 1 


-1 


-1 


-1 


1 


2 


2 








-1 


2 


2 





1 





V-i 





2 


-1 


2 



10.9 Find a basis for the kernel of 



(\ 


1 2\ 


-1 


-1 


V 1 


1 V 



10.10 Find a basis for the kernel of 



(l 





l\ 


2 


1 


-1 


V- 1 


1 


0/ 



10.11 Find the dimension of the kernel of 

'-(: !)■-(; 
-C :)•-(: 

10.12 Prove that the kernel of A is the kernel of 





10.13 If you know the kernel of a p x q matrix A, how do you find the dimension 
of the kernel of 





(A 


A 


A\ 




B = 


A 


A 


A 


? 




U 


A 


A) 





Image 

The image of a matrix is the set of vectors y of the form y = Ax for some vector 
x, written im A. 

10.14 Prove that the image of a matrix is the span of its columns. 
The image of 



is the span of 





(\ 


1 


2 


3) 


A = 





-1 


-2 


-3 




V- 1 
















(A 




{'•) 




(■' ) ' 




!'•) 






w 







10.15 Prove that the equation Ax = b has a solution x just when b lies in the 
image of A. 

Review problems 

10.16 Describe the image of the matrix 





(2 


o\ 


A = 





3 






0/ 



10.17 (a) Suppose that A is a 2 x 2 matrix which, when taking a vector x 
to the vector Ax, takes multiples of e\ to multiples of e\. Show that A is 
upper triangular. 

(b) Similarly, if A is a 3 x 3 matrix which takes multiples of e\ to multiples 
of ei, and takes linear combinations of e\ and e-i to other such linear 
combinations, then A is upper triangular. 

(c) Generalize this to R n , and use it to show that the inverse of an invertible 
upper triangular matrix is upper triangular. 

10.18 Prove that if a matrix is taller than it is wide (a "tall" matrix), then 
some vector does not belong to its image. 



10.19 Find a basis for the image of 



I— 1 


I— 1 


o\ 


-1 





-1 


V 2 


2 


h- 1 



Explain your answer. 

10.20 Find a basis for the image of 



Explain your answer. 

10.21 Find a basis for the image of 



Explain your answer. 

10.22 Find a basis for the image of 



Explain your answer. 

10.23 Find a basis for the image of 



Explain your answer. 

10.24 Find a basis for the image of 



( 2 


-6 


-6\ 




18 


18 


\° 








(o 


2 


2 


-1 2 


V 


1 2 



2 


h- 1 


3 




3 


-9 











/-3 


1 





3 


-1 







-1 






2 


4 







-12 







8 






Explain your answer. 



10.25 Find a basis for the image of 

( 1 1 
1 

V-l 2 

Explain your answer. 

10.26 Find a basis for the image of 

/-I 
1 
V-l 2 

Explain your answer. 

10.27 Find a basis for the image of 



Explain your answer. 

10.28 Find a basis for the image of 




Explain your answer. 

10.29 Find a basis for the image of 



Explain your answer. 

10.30 Find a basis for the image of 

A 

2 

V 



2 

V 



1 

-v 





4 




3 


-12 


-12 




-12 


-12 



/2 


-6 




-6 


18 


12 


V° 









-1 

0/ 



Explain your answer. 



Kernel and Image 

10.31 If A and B are matrices, and there is an invertible matrix C for which 
B = CA, prove that A and B have the same kernel. 

10.32 If A and B are two matrices, and B — CA for an invertible matrix C, 
prove that A and B have images of the same dimension. 

Theorem 10.3. For any matrix A, 

dim ker A + dim im A = number of columns. 



Proof. The image of A is the span of the columns. By lemma 8.1 on page 64, 
each pivotless column is a linear combination of earlier pivot columns. So 
the pivot columns span the image. Pivot columns are linearly independent: a 
basis. Each pivotless column contributes (in our algorithm) to our basis for the 
kernel. □ 



The matrix 



has echelon form 



A = 



U 





1 


-1 


Ci 


1 


1 




2 


-1 




1 


-1 








2 











not of U) are 


a basis 




The image has dimension 3 because there are 3 pivot columns. The kernel has 
dimension 1 , because there is one pivotless column. The pivotless column is not 
a basis for the kernel. It just shows you the dimension of the kernel. (In this 
example, the pivotless column isn't even in the kernel.) 



10.33 Find the rank of 



A) 
1 


Vo 



2 2\ 



2 2 

2 2/ 



and explain what this tells you about image and kernel. 



Review problems 



10.34 Find two matrices A and B, which have different images, but for which 
B = CA for an invertible matrix C. Prove that the images really are different, 
but of the same dimension. 

10.35 Prove that rank A = rank A* for any matrix A. 
Summary 

a. The kernel of a matrix is the set of vectors it kills. It is large just when 
linear equations Ax — b with one solution have lots of solutions (measures 
plurality of solutions when they exist). 

b. Our algorithm makes a basis for the kernel out of the pivotless columns. 

c. The image of a matrix is the stuff that comes out of it — the vectors b for 
which you can solve Ax = b (measures existence of solutions). 

d. The pivot columns are a basis of the image. 

Review problems 

10.36 Suppose that Ax = b. Prove that A(x + y) = b too, just when y lies in 
the kernel. So the kernel measures the plurality of solutions of equations, while 
the image measures existence of solutions. 

10.37 What is the maximum possible rank of a 4 x 3 matrix? A 3 x 5 matrix? 

10.38 If a 3 x 5 matrix A has rank 3 must the equation Ax = b have a solution 
x? Can it have more than one solution? If it has one solution, must it have 
infinitely many? 

10.39 As for the previous question, but with a 5 x 3 matrix A. 

10.40 If A = BC and B is 5 x 4 and C is 4 x 5, prove that det A = Q. 

10.41 Write down a 2 x 2 matrix A so that if I choose any vector x with positive 
entries, then the vector Ax also has positive entries, and lies between (but not 
on) the horizontal axis and a diagonal line. 

10.42 The Fredholm Alternative: for any matrix A and vector b, prove that 
just one of the following two problems has a solution: (1) Ax = b or (2) A t y = 
with b l y ^ 0. 

10.43 Prove that the image of AB is contained in the image of A. 

10.44 Prove that the rank of AB is never more than the rank of A or of B. 

10.45 Prove that the rank of a sum of matrices is never more than the sum of 
the ranks. 



10.46 Which of the following can change when you carry out forward elimina- 
tion? 

a. image, 

b. kernel, 

c. dimension of image, 

d. dimension of kernel? 

10.47 Prove that the rank of AB is no larger than the ranks of A and B. 



Eigenvectors 



Chapter 11 



Eigenvalues and Eigenvectors 



In this chapter, we study certain special vectors, called eigenvectors, associated to a 
square matrix. 



What are eigenvectors 




When a vector x is struck by a matrix A, it becomes a new vector Ax. Usually 
the new vector is unrelated to the old one. Rarely, the new vector might just 
be the old one stretched or squished; we will then call x an eigenvector of A. 
If we have a basis worth of eigenvectors, then the matrix A just squishes or 
stretches each one, and we can completely recover the matrix if we know the 
basis of eigenvectors and their eigenvalues. 

Eigenvalues and the Characteristic Polynomial 

An eigenvector a; of a square matrix A is a nonzero vector for which Ax = Xx 
for some number A, called the eigenvalue of x. The eigenvalue is the factor that 
the eigenvector gets stretched by. 

11.1 For each of the pictures in problem 3.16 on page 25, calculate the eigen- 
values of the associated matrix and draw on each face the directions that the 
eigenvectors point in. 

Lemma 11.1. A number X is an eigenvalue of a square matrix A (which is to 
say that there is an eigenvector x with that number as eigenvalue) just when 

det (A~ XI) = 0. 

Proof. Rewrite the equation Ax — Xx as (A — XI) x — 0. Recall that the 
equation Bx = (with B a square matrix) has a nonzero solution x just when 
B is not invertible, so just when det B = 0. Lets pick B to be A — A /; there is 
an eigenvector x with eigenvalue A just when det (A — XI) = 0. □ 

The expression det (^4 — A /) is called the characteristic polynomial of the 
matrix A. 

We can restate the lemma: 

Lemma 11.2. The eigenvalues of a square matrix A are precisely the roots of 
its characteristic polynomial. 



Figure 11.1: An 
eigenvector just gets 
stretched 
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The matrix 

A = 

has characteristic polynomial 



So the eigenvalues are A = 2 and A = 3. 



det 



'2-A 
. 




= (2 - A) (3 - A) . 



11.2 Prove that the eigenvalues of an upper (or lower) triangular matrix are 
the diagonal entries. For example: 





(\ 


2 


3\ 


A = 





4 


5 




\o 







4, A = 6. 







has eigenvalues A = 1, A = 4, A = 6. 

11.3 Find a 2 x 2 matrix A whose eigenvalues are not the same as its diagonal 
entries. 

11.4 Find 2x2 matrices A and B for which A + B has an eigenvalue which is 
not a sum of some eigenvalue of A with some eigenvalue of B. 

The set of eigenvalues of a matrix is called its spectrum. 
The matrix 

.1 j 



A = 



has characteristic polynomial det (A — XI) = A 2 + 1, which has no roots, so 
there are no eigenvalues (among real numbers A). 

11.5 Why is the characteristic polynomial a polynomial in A? 

11.6 What is the highest order term of the variable A in the characteristic 
polynomial of a matrix Al 



Appendix: Why the Fast Formula is So Slow 

We use the slow formula to calculate determinants when we compute out 
characteristic polynomials. Why not the fast formula? Lets try it on an 
example. Take 





(l 


1 2\ 


A = 


2 


3 




V 


1 4/ 



Lets hunt down the eigenvalues of A, by computing the characteristic polynomial 
as before. But this time, lets try the fast formula for the determinant, to find 
det (A — XI). We apply forward elimination to 



A — XI 




1 

3- A 
1 




to find 



2 

V i 



1 

3 — A 
1 




Add — y3^(row 1) to row 2, — j-z^(row 1) to row 3. 



■±=±gf 4^1 + A)- 1 

A 2-5 A+A 2 

-1 + A 




-1 + A 



Move the pivot \ 

/[ i^x] 



V o 



1 -4A + A 2 
-1 + A 



-1+A 



4 (-1 + A)" 1 

2-5 A+A 2 
-1+A 



/ 



Add 



A 

1-4 A+A 2 



(row 2) to row 3. 



/ 1-A 


V o 

Move the pivot \. 

fl i^x] 

Q 




1 -4A + A 2 
-1 + A 




4 (-1 + A)- 1 

A 3 -8 A 2 + 15 A-2 
1-4 A+A 2 



1 -4A + A 2 

-1 + A 




4 (-1 + A)- 1 

A 3 -8A 2 + 15A-2 
1 - 4 A + A 2 J 



The point: at each step, the expressions are rational functions of A, accumulating 
to become more complicated at each step. This is not any faster than the slow 
process, which gives: 



det (A-A/)=+(l-A)det( 3 



1 





=2-15A + 8A 2 -A 3 . 

Always use the slow process when searching for eigenvalues. There is actually a 
faster method to find eigenvalues of large matrices, but it is slower on small 
matrices, and we won't ever want to work with large matrices. 

Review problems 

11.7 Find the eigenvalues of the matrices 

.4=1 4 . : 2 U=( 2 3 U=(° »), fl _i; I t 



-2 




1 oj 



11.8 Prove that a square matrix A and its transpose A* have the same eigen- 
values. 

11.9 Prove that 

det (F^AF - A /) = det {A - A I) 

for any square matrix A, and any invertible matrix F. So the characteristic 
polynomial is unchanged by change of basis. 

11.10 If all of the entries of a square matrix are positive, are its eigenvalues 
positive? 

11.11 Are the eigenvalues of AB equal to those of BA1 

11.12 Give an example of 2 x 2 matrices A and B for which the eigenvalues of 
AB are not products of eigenvalues of A with those of B. 

11.13 What are the eigenvalues of 

(l 1 l\ 

A= 1 1 1 ? 

v i v 



The multiplicity of an eigenvalue Xj is the number of factors of A — Xj 
appearing in the characteristic polynomial. 

11.14 Suppose that the characteristic polynomial of some n x n matrix A 
splits into a product of linear factors. Prove that the determinant of A is the 
product of its eigenvalues (each taken with multiplicity) , by setting A = in 
the characteristic polynomial. 

11.15 From the previous exercise, if a 2 x 2 matrix A has eigenvalues and 1, 
what is its rank? 

11.16 Write out the characteristic polynomial of an n x n matrix A as 
det (A — XI) = s (A) - Sl (A)X + s 2 {A)X 2 + ■■■ + (-l) n s n (A)X n . 

a. Find s n (A). 

b. Prove that s Q (A) = det A. 

c. Prove that Sj(A) is a sum of products of precisely n — j entries of A. In 
particular, s n ^i(A) is a polynomial of degree 1 as a function of each entry 
of A. 

d. Use this to prove that s n ~i(A) = An + A 2 2 + • • • + A nn . (This quantity 
An + A 2 2 + • • • + A nn is called the trace of A). 

e. Prove that Sj = Sj(A) for any invertible matrix F, so the 
coefficients of the characteristic polynomial are unchanged by change of 
basis. 

f. Take a basis Ui, u 2l ■ ■ ■ , u n for which the vectors u r+ i,u r+2 , . . . , u n form 
a basis of the kernel, let F be the associated change of basis matrix, and 
look at F~ X AF. Prove that 



g. If A has rank r, prove that Sk(A) = for k < n — r. 

h. Write down two 2x2 matrices of different ranks with the same characteristic 
polynomial. 

How to find eigenvectors 

To find the eigenvectors of a matrix A: once you have the eigenvalues, pick each 
eigenvalue A, and find the kernel of A — XL 
The matrix 




for some invertible r x r matrix P, and some matrix Q. 




has eigenvalues A = 2 and A = 3. 



Lets start with A = 2: 




'->'-(" :)-(! "I 
■{• :) 

Our algorithm (from section 10) for finding the kernel yields a basis 

(-.') 

for the A = 2-eigenvectors. 

11.17 Do the same for A = 3. 

11.18 Find the eigenvectors and eigenvalues of 

3 
G 


Put it all together. How do we calculate the eigenvectors of 

;> 

a. Find the eigenvalues: 

= dct (A — X I) 
= (3-A)(l-A) 



So the eigenvalues are A = 3 and A = 1 . 
b. Find the eigenvectors: for each eigenvalue A, compute a basis for the kernel 
of A — XI. For A = 3: 

-ft - 2 ) 



The kernel of A — 3 / has basis 



The nonzero linear combinations of this basis are the eigenvectors with 
eigenvalue A = 3. 
For A = 1, 



A-I = 



3-1 
v 

,0 0, 



1 - 1 , 



The kernel of A — I has basis 



The nonzero linear combinations of this basis are the A = 1-eigenvectors. 
Review problems 

11.19 Without any calculation, what are the eigenvalues and eigenvectors of 




11.20 Find the eigenvalues and eigenvectors of 

A ■■ 



1 1 

,0 1, 



11.21 Find the eigenvalues and eigenvectors of 

*-(: ;) 

11.22 Find the eigenvalues and eigenvectors of 

1 2 (A 
A= 2 1 3 
\2 3/ 



11.23 Prove that every eigenvector of any square matrix A is an eigenvector of 
A^ 1 , of A 2 , of 3 A and of A — 71. How are the eigenvalues related? 

11.24 Forward elimination messes up eigenvalues and eigenvectors. Back 
substitution messes them up further. Give the simplest examples you can. 

11.25 What are the eigenvalues and eigenvectors of the permutation matrix of 
a transposition? 

11.26 What are the eigenvalues and eigenvectors of a 2 x 2 strictly lower 
triangular matrix? 

11.27 What are all of the real numbers that can occur as values of the 

(a) determinant 

(b) trace 

(c) pivots 

(d) eigenvalues 

of n x n permutation matrices? Justify your answers. 

11.28 Find a matrix with the given eigenvectors. 



11.29 Find a matrix with the given eigenvectors. 




11.30 Find a matrix with the given eigenvectors. 




11.31 Find a matrix with the given eigenvectors. 




11.32 Find a matrix with the given eigenvectors. 




11.33 Find a matrix with the given eigenvectors. 




11.34 Find a matrix with the given eigenvectors. 




11.35 Find a matrix with the given eigenvectors. 




11.36 Find a matrix with the given eigenvectors. 




11.37 Find a matrix with the given eigenvectors. 




11.38 Find a matrix with the given eigenvectors. 



11.39 Find a matrix with the given eigenvectors. 




11.40 Find a matrix with the given eigenvectors. 



I l I I l i I 

11.41 Find a matrix with the given eigenvectors. 




11.42 Find a matrix with the given eigenvectors. 



Chapter 12 



Bases of Eigenvectors 



In this chapter, we try (and don't always succeed) to organize eigenvectors into bases. 



Eigenspaces 

The X-eigenspace of a square matrix A is the set of vectors x for which (A — 
XI)x = (i.e. the kernel of A— XI). The eigenvectors are precisely the nonzero 
vectors in the eigenspace. In particular, if A is not an eigenvalue, then the 
A-eigenspace is just the vector. 

12.1 Prove that for any value A, the A-eigenspace of any square matrix is a 
subspace. 

Review problems 

12.2 Suppose that A and B are n x n matrices, and AB = BA. Prove that if 
x is in the A-eigenspace of A, then so is Bx. 

Bases of Eigenvectors 

Diagonal matrices are very easy to work with: 



Figure 12.1: An eigenspace with eigenvalue A — 2: anything you draw in that subspace get doubled. 
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Figure 12.2: A basis of eigenvectors of a matrix. Each vector starts off as a thickly drawn vector, 
and gets stretched into the thinly drawn vector. A negative stretching factor reverses the direction 
of the vector. We can recover the entire matrix A from this picture: if we know the directions of 
the basis vectors that are stretched and how much the matrix stretches vectors in each of those 
directions, we can find the matrix. 



each variable simply getting scaled by a factor. The next easiest are matrices 
that become diagonal when we change variables. 

Theorem 12.1 (Decoupling Theorem). //u 1 ,ti2,...,w n is a basis ofM. n , and 
each of Ui,«2, . . . ,u n is an eigenvector of a square matrix A, say Aui = 
XiUi,Au2 = A 2 M2, • ■ • , Au n = \ n u n , then 



F~ X AF 



V 



whe 



F = (ux U2 ... u 

is the change of basis matrix of the basis u\, Ua, . . . , u n 



The matrix F (or the basis u%, 112, ■ ■ ■ , u n ) diagonalizes the matrix A. 

We call this the decoupling theorem, because the transformation taking x 
to Ax is usually very complicated, mixing up the variables x\, x%, . . . ,x n in a 
tangled mess. But if we can somehow change the variables and make A into 
a diagonal matrix, then each of the new variables is just being stretched or 
squished by a factor A;, independently of any of the other variables, so the 
variables appear "decoupled" from one another. 

Proof. F takes e's to it's, A scales the u's, and then F -1 turns the scaled u's 
back into e's. So F~ x AFtj = A,ej, giving the j-th column of F~ 1 AF. So 



Ai 



F~ X AF 



\ 



□ 



12.3 Diagonalizc 



A = 







2 



1 







2 











3 







: ) 



We save a lot of time if we notice that: 

Theorem 12.2. Eigenvectors with different eigenvalues are linearly indepen- 
dent. 

This saves us time because we don't have to check to see if the eigenvectors 
we come up with are linearly independent, since we generate a basis for each 
eigenspace, and there are no relations between eigenspaces. 

Proof. Take a square matrix A. Pick some eigenvectors, say x\ with eigenvalue 
Ai, x 2 with eigenvalue A2, etc., up to some x p . Suppose that all of these 
eigenvalues Ai, A2, . . . , X p are different from one another. If we found a linear 
relation C\X\ = involving just one vector x\, we would divide by c\ to see 
that X\ = 0. But x\ ^ (being an eigenvector), so there is no linear relation 
involving just one eigenvector. Lets suppose we found a linear relation involving 
just two eigenvectors, x\ and x 2 , like C\X\ + c 2 x 2 = 0. We could just replace X\ 
by c\X\ and x 2 by c 2 x 2 to arrange a linear relation x\ + x 2 = 0. Since x\ is an 
eigenvector with eigenvalue Ai, we know that (A — Xi I) x\ = 0. Apply A — X\l 
to both sides of our relation to get (A2 — \\) x 2 — 0. Since the eigenvalues are 
distinct, we can divide by A2 — Ai to get x 2 = 0, again a contradiction. So there 
are no linear relations involving just two eigenvectors. 
Lets imagine a linear relation 



involving any number of eigenvectors, and see why that leads us into a contra- 
diction. If any of the terms are 0, just drop them, so we can assume that there 
are no terms, i.e. that all coefficients ci, c 2 , ■ . ■ , c p are nonzero. So we can 
rescale, replacing x\ by c\X\, etc., to arrange that our relation is now 



Applying A — Ai I to our linear relation: 

= (A - Ai I) {x-i, + x 2 + ■ ■ ■ + x p ) 
= (A2 - Ai) x 2 H h (X p - Ai) x p 

a linear relation with fewer terms. Since Ai ^ A2, the coefficient of x 2 won't 
become in the new linear relation unless it was already 0, so this new linear 
relation still has nonzero terms. In this way, each linear relation leads to a 
linear relation with fewer terms, until we get down to one or two terms, which 
we already saw can't happen. Therefore there are no linear relations among 

Xi,X2,-..,X n . □ 



c\xi + c 2 x 2 H h CpXp = 0, 



xx + x 2 -\ h x p = 0. 



12.4 Diagonalizc 



12.5 Diagonalizc 





'-I 


2 


2 


A = 


2 


2 






v-3 


-6 






/-I 


3 


-3\ 


A = 


-3 


5 


-3 




V-6 


6 





12.6 Prove that a square matrix is diagonalizable (i.e. diagonalized by some 
matrix) just when it has a basis of eigenvectors. 

So F diagonalizes A just when the change of coordinates y = F~ x x changes 
the matrix A into a diagonal matrix. 

12.7 Give an example of a matrix which is not diagonalizable. 

12.8 If A is diagonalized by F, say F~ 1 AF = A diagonal, then prove that A 2 
is also diagonalized by F. Apply induction to prove that all powers of A arc 
diagonalized by F. 

12.9 Use the result of the previous exercise to compute A wooo ° where 



Review problems 

12.10 Find the eigenvalues and eigenvectors of 

A = 



12.11 Find the matrix A which has eigenvalues 1 and 3 and corresponding 
eigenvectors 

Q(?) 

12.12 Imagine that a quarter of all people who are healthy become sick each 
month, and a quarter die. Imagine that a quarter of all people who are sick die 
each month, and a quarter become healthy. What happens to the dead people? 
Write a matrix A to show how the numbers h n , s n ,d n of healthy, sick and dead 
people change from month n to month n + 1. Diagonalizc A. What happens to 
the population in the long run? Prove your answer. (Keep in mind that no one 
is being born in this story.) 



12.13 Let A be the 5x5 matrix all of whose entries are 1. 

a. Without any calculation, what is the kernel of A? 

b. Use this to diagonalizc A. 

12.14 Lets investigate which 2x2 matrices are diagonalizable. 
a. Prove that every 2x2 matrix A can be written uniquely as 




for some numbers p, q, r, s. 

b. Prove that the characteristic polynomial of A is (p — A) 2 + s 2 — q 2 — r 2 . 

c. Prove that A has two different eigenvalues just when q 2 + r 2 > s 2 . 

d. Prove that any 2x2 matrix with two different eigenvalues is diagonalizable. 

e. Prove that any 2x2 matrix with only one eigenvalue is diagonalizable 
just when it is diagonal. 

f. Prove that any 2x2 matrix with no eigenvalues is not diagonalizable. 

12.15 Find all real eigenvalues of the matrix 




For each eigenvalue, find a basis of its eigenvectors. 
12.16 Find all real eigenvalues of the matrix 




For each eigenvalue, find a basis of its eigenvectors. 
12.17 Find all real eigenvalues of the matrix 




For each eigenvalue, find a basis of its eigenvectors. 
12.18 Find all real eigenvalues of the matrix 




For each eigenvalue, find a basis of its eigenvectors. 



12.19 Find all real eigenvalues of the matrix 

For each eigenvalue, find a basis of its eigenvectors. 

12.20 Find all real eigenvalues of the matrix 

For each eigenvalue, find a basis of its eigenvectors. 

12.21 Find all real eigenvalues of the matrix 

•■(-: :) 

For each eigenvalue, find a basis of its eigenvectors. 

12.22 Find all real eigenvalues of the matrix 



For each eigenvalue, find a basis of its eigenvectors. 

12.23 Find all real eigenvalues of the matrix 

•■(: -.') 

For each eigenvalue, find a basis of its eigenvectors. 

12.24 Find all real eigenvalues of the matrix 

For each eigenvalue, find a basis of its eigenvectors. 

12.25 Find all real eigenvalues of the matrix 



For each eigenvalue, find a basis of its eigenvectors. 
12.26 Find all real eigenvalues of the matrix 

A -{~* 4)- 

For each eigenvalue, find a basis of its eigenvectors. 



Summary 



Linear algebra has two problems: 

a. Solving linear equations Ax — b for the unknown x. This problem is truly 
linear. It has a solution x whenever b lies in the image, and the solution x 
is unique up to adding on vectors from the kernel. 

b. Find eigenvectors and eigenvalues Ax — Xx. This problem is nonlinear, in 
fact quadratic, since A and x are multiplied by one another. The nonlinear 
part is finding the eigenvalues A, which are the roots of the characteristic 
polynomial det (A — XI). There is an eigenspace of solutions x for each A, 
and finding a basis of each eigenspace is a linear problem. If we get lucky 
(which doesn't always happen), then the eigenvectors might form a basis of 
M. n , diagonalizing A. 

Table 12.1: Invertibility criteria (Strang's nutshell [5]). A is n x n. 
U is any matrix obtained from A by forward elimination. 



§ 


Invertible Just When . . . 




5 


Gauss- Jordan on A yields /. 




5 


U is invertible. 




5 


Pivots lie all the way down the diagonal. 


5 


U has no zero rows 




5 


U has n pivots. 




5 


Ax = b has a solution x for each b. 




5 


Ax = b has exactly one solution x for 


each b. 


•5 


Ax = b has exactly one solution x for 


some b. 


•5 


Ax = only for x = 0. 




5 


A has rank n. 




7 


A 1 is invertible. 




7 


det A ^ 0. 




9 


The columns are linearly independent 




9 


The columns form a basis. 




9 


The rows form a basis. 




10 


The kernel of A is just the vector. 




10 


The image of A is all of K n . 




11 


is not an eigenvalue of A. 





12.27 Take each of the criteria in table 12.1, and describe an analogous criterion 
for showing that A is not invertible. For example, instead of det A ^ 0, you 
would write det A = 0. Make sure that as many as possible of your criteria 
express the failure of invertibility in terms of the rank r of the matrix A. For 



example, instead of turning 

U has no zero rows 

into 

U has a zero row, 

you should turn it into 

U has n — r zero rows. 



Orthogonal Linear Algebra 



Chapter 13 



Inner Product 



So far, we haven't thought about distances or angles. The elegant algebraic way to 
describe these geometric notions is in terms of the inner product, which measures 
something like how strongly in agreement two vectors are. 



Definition and Simplest Properties 

The inner product (also called the dot product or scalar product) of two vectors 
x and y in K" is the number 



(x, y) = xxyt + x 2 y 2 H 1- x n y„ 



The vectors 



have inner product 












> y = 


(i) 


W 







In 



(!,!/> = (l)(4) + (0)(5) + (2)(6) 
= 16. 




13.1 Prove that (Aej,ei) = Aij. 

13.2 Let P be a permutation matrix. Use the result of problem 13.1 to prove 
that P- 1 =P t . 



Recall that the transpose A* of a matrix A is the matrix with entries 
in 

13.3 Prove that (x, y) — x*y. 



A*j — Aji, i.e. with rows and columns switched 



Vectors u and v are perpendicular if (u, v) — 0. The length of a vector x in 
is ||x|| = \J (x, x). This agrees in the plane with the Pythagorean theorem: 
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Figure 13.1: The Pythagorean theorem. Rearrange the 4 triangles into 2 rectangles to find the 
area of all 4 triangles. Add the area of the small white square. 



if x = ( then we can draw a: as a point of the plane, and the length along x 

W 

is Va 2 + b 2 . 

13.4 Prove that for any vectors u and v, with u^O, the vector 



is perpendicular to u. 



Review problems 



13.5 How many vectors x in R™ have integer coordinates x\, X2, ■ ■ ■ , x n and 
have 

(a) \\x\\ = 0? 

(b) 11*11 = 1? 

(c) ||*|| = 2? 

(d) ||*|| = 3? 

13.6 a. What is wrong with the clock? 



b. At what times of day are the minute and hour hands of a properly func- 
tioning clock 

a) perpendicular? 

b) parallel? 

(The answer isn't very pretty.) 

13.7 Prove that (Ax, y) = (*, A t y) for vectors x in K 9 , y in K p and A any px q 
matrix. 



Symmetric Matrices 

A matrix A is symmetric if A 1 = A. 

13.8 Which of the following are symmetric? 

1 2 

3 4 

4 5 

13.9 Prove that an n x n matrix is symmetric just when 

(Ax, y) = (x, Ay) 

for x and y any vectors in R n . 

Clearly a symmetric matrix is square. 

13.10 Prove that 

a. The sum and difference of symmetric matrices is symmetric. 

b. If A is a symmetric matrix, then 3^4 is also a symmetric matrix. 

13.11 Give an example of a pair of symmetric 2x2 matrices A and B for 
which AB is not symmetric. 

Review problems 

13.12 For which matrices A is the matrix 

B = 




symmetric? 

13.13 If A is symmetric, and F an invertible matrix, is FAF -1 symmetric? If 
not, can you give a 2 x 2 example? 

13.14 If A and B are symmetric, is AB + BA symmetric? Is AB — BA 
symmetric? 

Orthogonal Matrices 

A matrix F is orthogonal if F l F — I. In problem 13.2, you proved that 
permutation matrices are orthogonal. 

13.15 Which of the following are orthogonal? 





Orthogonal matrices are important because they preserve inner products: 
13.16 Prove that a matrix is orthogonal just when 



for x and y any vectors. 

Clearly any orthogonal matrix is square. 

13.17 Prove that 

a. The product of orthogonal matrices is orthogonal. 

b. The inverse of an orthogonal matrix is orthogonal. 

13.18 If F is orthogonal, and c is a real number, prove that cF is also orthogonal 
only when c — ±1. 

13.19 Which diagonal matrices are orthogonal? 

13.20 Prove that the matrices 



are orthogonal. Give an example of an orthogonal 2x2 matrix not of this form. 

13.21 Give an example of a pair of orthogonal 2x2 matrices A and B for 
which A + B is not orthogonal. 

13.22 By expanding out the expressions (x + y, x + y) using the properties of 
inner products, express the inner product (x, y) of two vectors in terms of their 
lengths. Use this to prove that a matrix A is orthogonal just when 



for any vector x. 
Orthonormal Bases 

Some bases are much easier to use than others. A basis ui,u 2 , ■ ■ ■ ,u n is 
orthonormal if 



(Fx, Fy) = (x,y) 




\\Ax\\ = \\x 




The standard basis is orthonormal 



The basis 




is orthonormal. 



Why Are Orthonormal Bases Better Than Other Bases? 

Take any basis ui, u 2 , ■ ■ ■ , u n for R". Every vector x in K™ can be written as a 
linear combination 

x = c x u x + c 2 u 2 H h c n u n . 

How do you find the coefficients ci, c 2 , . . . , c„? You apply elimination to the 
matrix 

{ui u 2 ... u n xj ■ 

This is a big job. But if the basis is orthonormal then you can just read off the 
coefficients as 

ci = (x, Mi) , c 2 = (x, u 2 ) , • • • , c„ = (x, u„) . 

13.23 Prove that if «i, u 2 , . ■ . , u n is an orthonormal basis for R" and x is any 
vector in R™ then 

x = (x, Ul) Ml + (x, u 2 )u 2 ^ h (x, u n ) u n . 

How Do We Tell If a Basis is Orthonormal? 

Proposition 13.1. A square matrix is orthogonal just when its columns are 
orthonormal. 



Proof. Write the matrix as 

F = (^ui u 2 



Calculate 



Ft : 



u 2 



Ul u 2 



\Un7 

u 2 'ui u 2 u 2 



U^Ux u n t u 2 



\ 

( (ui,ui) 
{u 2 ,ui) 



(«1,«2> 
{U2,U 2 ) 



(u n ,ui) (u n ,u 2 ) 



V 



in) 



u 2 u r , 



u n u n 



J 

(ui,u n )\ 
{u 2 ,u n ) 

/ 



□ 



The original vectors. 




Project the second 
vector perpendicular to 
the first. 



Projected. 



Shrink/stretch all 
vectors to length 1. 




Done: orthonormal. 



Figure 13.2: The Gram— Schmidt process 



13.24 Is the basis 




orthonormal? Draw a picture of these two vectors. 



13.25 Prove that a square matrix is orthogonal just when its rows are orthonor- 
mal. 



Gram-Schmidt Orthogonalization 



The idea: if I start with a basis V\, «2 of K 2 which is not orthonormal, I can fix 
it up (as in figure 13.2). 

The formal definition: given any linearly independent vectors v%, i?2, . . . , v p 



(as input), the output are orthonormal vectors u\, u 2 , ■ ■ ■ , u p : 

Wi = Vl, 

(V2,Wl) 

W 2 = V 2 - 7 r Wi, 

(Wi,Wi) 

(v 3 ,W!) {v 3 ,w 2 ) 

W 3 =V 3 - Wi - r W 2 , 

{Wi,Wi} {w 2 ,w 2 ) 
\ - (Vj,Wi) 

1 



Each Wj is just Vj with all parts "pulled off" that head in the directions of 
previous Wi's (the directions we are already finished with). At the final step, 
each Uj is just Wj rescaled to unit length. We say that we are orthogonalizing 
the vectors Vi , v 2 , . . . , v p . 

13.26 Orthogonalizc 











Vl = 




... 


:•,) 




v o y 







13.27 Orthogonalizc 




13.28 Orthogonalizc 




and then draw pictures explaining the process. 
13.29 Orthogonalizc 















Vl = 


I 


, v 2 = 


K 



















13.30 Prove that if vi, v 2 , ■ ■ ■ , v p are linearly independent vectors, then each 
step of Gram-Schmidt makes sense (no dividing by zero), and the resulting 
Mi, u 2 , . . . , Up are an orthonormal basis for the span of Vi, v 2 , . . . , v p . 



13.31 Prove that any set of vectors, all of unit length, and perpendicular to 
one another, is contained in an orthonormal basis. 

13.32 If u and v are two vectors in R™, and every vector w which is perpendic- 
ular to u is perpendicular to v, then v = au for some number a. 

Review problems 

13.33 What happens to a basis when you carry out Gram-Schmidt, if it was 
already orthonormal to begin with? 

13.34 Orthogonalize 




13.35 Orthogonalize 




13.36 Orthogonalize 




13.37 Orthogonalize 




13.38 Orthogonalize 




13.39 Orthogonalize 




13.40 Orthogonalize 




13.41 Orthogonalize 




13.42 Orthogonalize 




13.43 Orthogonalizc 



13.44 Orthogonalizc 



13.45 Orthogonalizc 



13.46 Orthogonalizc 



Chapter 14 



The Spectral Theorem 



Finally, our goal: we want to prove that symmetric matrices can be made into diagonal 
matrices by orthogonal changes of variable. 



Statement and Proof 

Proposition 14.1 (The Minimum Principle). Let A be a symmetric n x n 
matrix. The function 

Q(x) = (Ax, x) 

is a quadratic polynomial function. Restrict x to lie on the sphere of unit length 
vectors. Then Q(x) reaches a minimum among all vectors on that sphere at 
some vector x = u. This vector u is an eigenvector of A. 

Proof. In the appendix to this chapter, we prove that the minimum occurs. So 
there is some vector i = mso that (Ax, x) > (Au, u) for any x of unit length. 
Fixing u, consider the quadratic function 

H(x) = (Ax, x) — (Au, u) (x, x) . 

For x of unit length, (x, x) = 1, so 

H(x) = (Ax,x) - (Au,u) 
> 0. 

But if we scale x, say to ax, clearly H (x) is quadratic in x, so 

H(ax) = a 2 H(x). 

So rescaling, we find that H(x) > for any vector x, of any length. Pick w any 
vector perpendicular to u. For any number t: 

< H(u + tw) 
= (A(u + tw) , u + tw) — (Au, u) (u + tw,u + tw) 
= (Au, u) + 2t (Au, w) + t 2 (Aw, w) - (Au, u) (l + t 2 (w, w)) 
= 2t (Au, w) + t 2 H(w) 
= t (2(Au,w) +tH(w)) . 
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First lets try t positive, so we can divide by t, and find < 2 (Au, w) + tH(w). 
Let t to go to zero, to see that < (Au, w). Next try t negative, and divide by t 
and then let t go to zero, and see that > (Au, w). Therefore (Au,w) — 0. So 
every vector w perpendicular to u is also perpendicular to Au. By problem 13.32 
on page 120, Au is a multiple of u, so u is an eigenvector. □ 

14.1 If two eigenvectors of a symmetric matrix have different eigenvalues, prove 
that they are perpendicular. 

Theorem 14.2 (Spectral Theorem). Each symmetric matrix A is diagonalized 
by an orthogonal matrix F. The columns of F form an orthonormal basis of 
eigenvectors. We say that F orthogonally diagonalizes A. 

Proof. Start with a unit eigenvector u\, given by the minimum principle. Take 
any orthonormal basis u±,U2, ■ ■ ■ ,u n that starts with this vector, and let F 
be the matrix with these vectors as columns. Replace A with F l AF. After 
replacement, A has e\ as eigenvector: Aei — Aei, so the first column of A is 
Aei. Because A is symmetric, we see that 

A ■■ 

with B a smaller symmetric matrix. By induction on the size of matrix, we can 
orthogonally diagonalize B. □ 

The previous exercise is vital for calculations: to orthogonally diagonal- 
ize, find all eigenvalues, and for each eigenvalue A find an orthonormal basis 
Ui,u%, . . . of eigenvectors of that eigenvalue A. All of the eigenvectors of all of 
the other eigenvalues will automatically be perpendicular to Ux, U2, ■ ■ ■ , so put 
together they make an orthonormal basis of R™ . 

14.2 Find a matrix F which orthogonally diagonalizes the matrix 

A = 

by finding the eigenvectors U\,u 2 and eigenvalues Ai, A2. 

14.3 Prove that a square matrix is orthogonally diagonalizable just when it is 
symmetric. 

An elegant (but longer) proof of the spectral theorem can be made along 
the following lines. Once we have used the minimum principle to find one 
eigenvector u\, we can then look among all unit length vectors x perpendicular 
to Ui, and see which of these vectors has smallest value for (Ax,x). Call that 
vector it 2 . Look among all unit vectors x which are perpendicular to both u\ 
and ui for one which has the smallest value of (Ax, x), and call it 1*3, etc. This 
recipe will actually generate the eigenvectors for us, although it isn't easy to 
use either by hand or by computer. 



Review problems 

14.4 Find a matrix F which orthogonally diagonalizes the matrix 

-('.-') 

14.5 Find a matrix F which orthogonally diagonalizes the matrix 



/4 







\ 3/ 
14.6 Find a matrix F which orthogonally diagonalizes the matrix 



14.7 Find a matrix F which orthogonally diagonalizes 



A 



14.8 Let 



25 



24 
25 



(l 



A = 



V 



y 



What are all of the orthogonal matrices F for which F l AF is diagonal with 
entries increasing as we move down the diagonal? 

14.9 If A is symmetric, prove that A 2 has the same rank as A. 

14.10 If A is n x n, prove that A 1 A has no negative eigenvalues, and its 
eigenvalues arc all positive just when A is invertible. 



Quadratic Forms 

A quadratic form is a polynomial in several variables, with all terms being 
quadratic (like x 2 or xy). In particular, no linear or constant terms can appear 
in a quadratic form. 



Symmetrizing 

If we have a quadratic form in variables x\ and x 2 , we can write it more 
symmetrically; for example: 

1 1 

x\x 2 = -Xix 2 + -x 2 xi. 

We just leave alone a term like x\, so for example 

x\ + 8xix 2 = x\ + 4.t 1 x 2 + Ax 2 X\. 

14.11 Symmetrize: 
2 

a. x 2 

b. x\ + x\ 

c. x\ + 3xix 2 

d. xi (xi + x 2 ) 



Making a matrix 

Pluck out the quadratic terms in the polynomial to make a matrix. For example: 
ax\ + bxix 2 + cx\ = ax\ + ^ x\x 2 + ^ x 2 xi + cx\ 

becomes 




More generally, AijXiXj becomes A = (Aij). Because we symmetrized, the 
matrix is symmetric. 

14.12 Make matrices for 
2 

a. x 2 

b. x\ + x\ 

c. x\ + I x x x 2 + I X 2 Xl 

d. x\ + \x x x 2 + \x 2 x\ 



Diagonalizing 

Diagonalize our matrix, by orthogonal change of variables. Then the same 
orthogonal change of variables will simplify our quadratic form, turning it into 
a sum of quadratic forms in one variable each. For example, take the quadratic 
form 



Symmetrize: 
The associated matrix is 



23 x\ + 72 X!X 2 + 2 x\. 



2Zx\ + 36 21X2 + 36x 2 xi + 2x 2 . 



A = 



We let the reader check that A is orthogonally diagonalized by 

F = 

so that 

F t AF = 

We also let the reader check that if we take new variables y, defined by y = F f x, 
i.e. by x — Fy, then the same quadratic form is 

-25y 2 + 50?/ 2 . 

Theorem 14.3 (Decoupling Theorem). Any quadratic form in any number of 
variables becomes a sum of quadratic forms in one variable each, after a change 
of variables x = Fy given by an orthogonal matrix F. 

The quadratic form is diagonalized by the orthogonal matrix. 

Proof. The problem comes from the mixed terms, like X\X<i- Symmetrize and 
write a symmetric matrix A out of the coefficients. Then the quadratic form 
is J^ij AijXiXj — (Ax,x). Diagonalize A to A = F t AF. Let y — F t x. Then 

x = Fy, so 

(Ax,x) = (AFy, Fy) 
= {F t AFy,y) 
= {Ay,y) 

= \ 1 y{ + --- + \ n y 2 n . 

□ 

Review problems 



14.13 Diagonalize 



(a) 


2,x x 2 - 


2xix 2 + 3x 2 2 


(b) 


%x x 2 + 


18xix 2 + 9x 2 2 


(c) 


llX! 2 


f 6xix 2 + 3x 2 2 


(d) 


3a;i 2 + 


4xix 2 + 6x 2 2 


(e) 


8 Xl 2 - 


12xix 2 + 3x 2 2 


(f) 


10xi 2 


f 6xix 2 + 2x 2 2 


(g) 


4X1X2 


+ 3x 2 2 


(h) 


6xx 2 - 


12xix 2 + 11 x 2 2 


(i) 


10X! 2 


- 12 Xix 2 + 5 x 2 2 


(j) 


-2X! 2 


- 4X!X 2 + x 2 2 


(k) 


2xi 2 - 


6xix 2 + 10 x 2 2 


(1) 


-2X! 2 


+ 6xix 2 + 6x 2 2 




(a) (b) El- (c) Pair of lines (d) Hyper- 
Circle lipse bola 

Figure 14.1: Some examples of solutions of quadratic equations 



Application to Quadratic Equations 

In the plane, with two variables x\ and x 2 , a quadratic equation Q(x) = c 
(with Q(x) a quadratic form and c a constant number) cuts out a circle, ellipse, 
hyperbola, pair of lines, single line, point, or empty set. The quadratic equation 

4 X1X2 + 3 x\ = 

involves the quadratic form with matrix 




Its eigenvalues are A = — 1 and A = 4. So we can change variables (somehow) 
to get to 

-x\ + Ax\ = 0. 

This is just 

xi = ±2x 2 , 

a pair of lines intersecting at a point. Since the change of variables is linear, the 
original quadratic equation also cuts out a pair of lines intersecting at a point. 
The equation 

x\ + 4xix 2 + x\ = 1 
contains the quadratic form with matrix 




The eigenvalues are A = — 1 and A = 3. So after a linear change of variables, we 
get 

~\~ 3 iz?2 — 1* 

(The right hand side is a constant, so doesn't change.) 
It is well known that an equation of the form 

a x\ + b x\ = 1 

with a and b of different signs is a hyperbola, while if a and b have the same 
signs then it is an ellipse. So our last example must be a hyperbola. Warning: 
until you diagonalize the associated matrix, and look at the eigenvalues, you 
can't easily see what shape a quadratic equation cuts out. You can't just look 
at whether the coefficients are positive, or anything obvious like that. 




(a) Positive definite (b) Positive semidef- (c) Indefinite (d) Negative defi- 
inite nite 



Figure 14.2: Various behaviours of quadratic forms in two variables 

Review problems 

14.14 Just by finding eigenvalues (without finding eigenvectors), determine 
what geometric shape (ellipse, hyperbola, pair of lines, line, empty set) the 
following are: 

a. 5 x\ + 2 x\ — 4:Xix 2 = 1 

b. 3x1 + &xix 2 + 3x1 = 1 

c. x\ — Ax\x 2 + — ~1 

d. 6x1X2 + 8x2 = 1 

e. x\ — § X\x 2 + 9 x\ = 

f. 4 xix 2 — 3 x\ — 6 x\ = 1 

14.15 What more can you do to normalize a quadratic form if you allow 
arbitrary invertible matrices instead of orthogonal ones? 

Positivity 

A quadratic form Q(x) is positive definite if Q{x) > except if x = 0. (Clearly 
if x = then Q(x) = 0.) For example, Q(x) = x 1 is a positive definite quadratic 
form on R, while Q(x) — x\ + x\ is positive definite on R ra . But it is not at all 
clear whether Q{x) = %x\ — \2x\x 2 + 11 x\ is positive definite, because it has 
positive terms and negative ones. As in figure 14.2, we can also define positive 
semidefinite forms (Q(x) > 0), negative definite forms (Q(x) < for 1 ^ 0), 
and indefinite forms (not positive semidefinite or negative semidefinite), but 
they are less important. 

Lemma 14.4. A quadratic form Q(x) = (x,Ax) (with a symmetric matrix A) 
is positive definite just if all of the eigenvalues of A are positive. 

Proof. Let Ai, A2, . . . , A„ be the eigenvalues, and change variables to y = F~ x x, 
so x — Fy, to diagonalize the quadratic form: 

Q = Ai y\ + A 2 y\ + • ■ ■ + A„ y 2 n . 

Suppose that all of the eigenvalues are positive. Clearly this quantity is then 
positive for nonzero vectors y, because each term is positive or zero, and at 
least one of y\ , y2, ■ ■ ■ , y n is not zero, so gives a positive term. 



On the other hand, if one of these Xj is negative, then take y = ej, and you 
get Q < but x = Fy = Fej ^ 0. □ 



Review problems 

14.16 Which of the quadratic forms in problem 14.13 on page 127 are positive 
definite? 

Appendix: Continuous Functions and Maxima 

There is one gap in our proof of the spectral theorem for symmetric matrices: 
we need to know that a quadratic function on the sphere in K™ has a maximum. 
This appendix gives the proof. Warning: students are not required or expected 
to work through this appendix, which is advanced and is included only for 
completeness. A sequence of numbers x\, X2, ■ ■ ■ converges to a number x if, in 
order to make Xj stay as close as we like to a;, we have only to ensure that j is 
kept large enough. A sequence of points 



if x\, X2, ■ ■ • converges to x and yi, y%, . . . converges to y. Similarly, a sequence 
of points in K n converges if all of the coordinates of those points converge. 

Any sequence of increasing real numbers, all of which are bounded from 
above by some large enough number, must converge to something. This fact is 
a property of real numbers which we cannot prove without giving an explicit 
and precise definition of the real numbers; see Spivak [4] for the complete story. 
We will just assume that this fact is true. 

A function / (x) of any number of variables x±, X2, ■ ■ ■ , x n (writing x for 



as a point of K") is continuous if, in order to get f(y) to stay as close to f(x) 
as you like, you have only to ensure that y is kept close enough to x. 

If two numbers are close, their sums, products and differences are clearly 
close. The reader should try to prove: 

Lemma 14.5. The function f (x) — X\ is continuous. Constant functions 
are continuous. The sum, difference and product of continuous functions is 
continuous. 




converges to a point 




X = 



\XnJ 



Corollary 14.6. Any polynomial function in any finite number of variables is 
continuous. 

Proof. Induction on the degree and number of terms of the polynomial. □ 

A ball in K™ is a set B consisting of all points closer than some distance to 
some chosen point (called the center of the ball) . The distance is called the 
radius. A closed ball includes also the points of distance equal to the radius (an 
apple with the skin), while an open ball does not include any such points, only 
including points of distance less than the radius (an apple without the skin) . 

A set 5* C W 1 is bounded if it lies in a ball (open or closed). 

A set S C W 1 is called closed if every point x of W l not belonging to S can 
be surrounded by an open ball not belonging to S. 

A closed box is the set of points x — (xi, . . . , x n ) for which each Xj lies 
in some chosen interval, aj < Xj < bj. An open box is the same but with 
dj < xj < bj. 

Lemma 14.7. A closed ball is a closed set, as is a closed box. 

Proof. Given a closed ball, say of radius r, take any point p not belonging to it, 
say of distance R from the center, and draw an open ball of radius R — r about 
p. By the triangle inequality, no point in the open ball lies in the closed ball. 

Given a closed box, and a point p not belonging to it, there must be some 
coordinate of p which does not satisfy the inequalities defining the closed 
box. For example, suppose that the box is cut out by inequalities including 
&1 < X\ < b\, and p fails to satisfy these bounds because p\ > b\. Then every 
point q closer to p than p\ — b\ will still fail: q\ > b\. So then a ball of radius 
pi — bi around p will not overlap the closed box. □ 

Theorem 14.8. Every infinite sequence of points in a closed, bounded set has 
a convergent subsequence. 

Proof. Suppose that the set is a box. Cover the box with a finite number of 
small closed boxes (perhaps overlapping). There are infinitely many points 
xi,x 2 , ■ ■ ■ , and only finitely many of the small boxes, so there must be infinitely 
many Xj lying in the same small box. 

Similarly, subdivide that small box into much smaller closed boxes. Re- 
peating, we find a sequence of closed boxes, like Russian dolls, each contained 
entirely in the previous one, with infinitely many Xj in each. We get to choose 
how small the boxes are going to be at each step, so lets make them get much 
smaller at each step, with side lengths decreasing as rapidly as we like. Pick out 
one of these Xi points, call it yj, from the j-th nested box, as the point with 
the smallest possible coordinates among all points of that box. The sequence of 
points t/i,t/2, • • • must converge, since all of the coordinates of the point yj are 
constrained by the box yj lies in, and each coordinate only increases with j. 



If we face a closed, bounded set S, which is not a closed box, then find a 
closed box B containing it, and repeat the argument above. The problem is 
to ensure that the limit x of the sequence constructed belongs to the set S. 
Even if not, it certainly belongs to B. Since S is closed, if x does not belong 
to S'jthen there must be an open ball around x not containing any points of S. 
But that open ball can not contain any of the points in the nested boxes, and 
therefore x cannot be their limit. □ 

Theorem 14.9. Every continuous function f on a closed, bounded set attains 
a maximum and a minimum. 

Proof. For the moment, lets suppose that our closed, bounded set is just a 
closed box. Suppose that / has no maximum. So the values of / can get larger 
and larger, but never peak. Let M be the smallest positive number so that / 
never exceeds M; if there is no such number let M = oo. By definition, / gets 
as close to M as we like (which, if M = oo, means simply that / gets as large 
as we like), but never reaches M. Let x\, X2, £3, • ■ • be any points of the closed 
bounded set on which / (xj) approaches M. Taking a subsequence, we find Xj 
approaching a limit point x, and by continuity / (xj) must approach f(x), so 
f(x)=M. 

So every continuous function on a closed bounded set has a maximum. If 
/ is a continuous function on a closed bounded set, then —/is too, and has a 
maximum, so / has a minimum. □ 



Chapter 15 



Complex Vectors 



The entire story so far can be retold with a cast of complex numbers instead of real 
numbers. Most of this is straightforward. But there turns out to be an important 
twist in the complex theory of the inner product. The minimum principle doesn't 
make any sense in the setting of complex numbers, and the spectral theorem as it was 
stated just isn't true any more for complex matrices. Moreover, the natural notion 
of inner product itself is quite different for complex vectors — this new notion leads 
directly to the complex spectral theorem. 



Complex Numbers 

A complex number is a pair (a;, y) of real numbers. Write 1 to mean the pair 
(1,0), and i to mean (0, 1). Addition is defined by the rule 

(x,y) + (X,Y) = (x + X,y + Y), 

subtraction by 

(x,y)-(X,Y) = (x-X,y-Y), 

and multiplication by 

(x, y) {X, Y) = (xX - yY, xY + yX) . 

We henceforth write any pair as x + iy. We call x the real part and y 
the imaginary part. When working with complex numbers, we draw them as 
points of the a;y-plane, which we call the complex plane. Complex numbers are 
associative, commutative and distributive, and every nonzero complex number 
z = x + iy has a reciprocal: 

1 x — iy 
z x 2 + y 2 

You can easily check all of this, but you may assume it if you prefer. 
Polar Coordinates 

Trigonometry tells us that any point (x, y) of the plane can be written as 
x = r cos 9,y = r sin 9 in polar coordinates. Therefore 

x + iy = r cos 9 + ir sin 8, 
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The number r is called the modulus of the complex number (written \z\ if z 
is the complex number) . The angle 9 is called the argument of the complex 
number (written arg z if z is the complex number). The modulus is the distance 
from 0, or the length if think of (x, y) as a vector. 

Theorem 15.1 (de Moivre). Under multiplication of complex numbers, moduli 
multiply, while arguments add. Under division of complex numbers, moduli 
divide, while arguments subtract. 

Proof. Let z and w be two complex numbers. Write them as z = r cos a + 
ir sin a and w = p cos /3 + ip cos /3. Then calculate 

zw = rp [(cos a cos /3 — sin a sin j3) + i (sin a cos /3 + cos a sin /?)] . 

A trigonometric identity: 

cos a cos /3 — sin a sin /3 = cos (a + f3) 
sin a cos /3 + cos a sin /3 = sin (a + /?) . 

Division is similar. □ 

15.1 Explain why every complex number has a square root. 

The conjugate z of a complex number z — x + iy is the number z = x — iy. 

15.2 If z is a complex number, prove that \z\ 2 = zz 

We write C for the set of complex numbers, and C™ for the set of vectors 

f Zl \ 

Z2 



VnJ 

with each of zi, z 2 , ■ ■ ■ , z n a complex number. We won't go through the effort 
of translating the theorems above into complex linear algebra, except to say 
that all of the results before chapter 13 (on inner products) are still true for 
complex matrices, with identical proofs. 



Review problems 

15.3 Draw the point z = — \ + |j on the plane. Draw z, 2z, z 2 , ^. 

15.4 Draw w = 1 + 2i and z = 2 + i, and draw z + w, zw, —. 



15.5 The unit disk in the complex plane is the set of complex numbers of 
modulus less than 1. The unit circle is the set of complex numbers of modulus 
1. Draw the unit circle. Pick z a nonzero complex number, and consider its 
integer powers z n (n ranging over the integers). Prove that either infinitely 
many of these powers lie inside the unit disk, or all of them lie on the unit 
circle. 



Use de Moivre's theorem to show that z% = 1. These are the so-called n roots 
of 1. Why are they all different for fc = 0, 1, . . . ,n — 1? Draw the 3 rd roots of 1 
and (in another colour) the 4 th roots of 1. 

15.7 With pictures and words, explain what you know about 



b. z 2 if \z\ > 1, 

c. z if \z\ = 1, 

d. zw if \z\ = \w\ = 1. 

Complex Linear Algebra 

The main differences between real and complex linear algebra are (1) eigenvalues 
and (2) inner products. We will consider inner products soon, but first lets 
consider eigenvalues. 

Theorem 15.2. Every square complex matrix has a complex eigenvalue. 

Proof. The eigenvalues are the roots of the characteristic polynomial det (A — X I) ; 
a polynomial with complex number coefficients in a complex variable A. The 
existence of a complex number root of any complex polynomial is proven in the 
appendix. □ 

It may be that there are not very many eigenvectors. 



as a complex matrix still only has one eigenvalue, and (up to rescaling) one 
eigenvector. Two linearly independent eigenvectors are just what we need to 
diagonalize a 2 x 2 matrix. Clearly we cannot diagonalize A. So complex 
numbers don't resolve all of the subtleties. 

15.8 Find the (complex) eigenvalues and eigenvectors of 



15.6 Let 




a 



z + z, 





and write down a matrix F which diagonalizes A. Moral of the story: even a 
matrix like A, which has only real number entries, can have complex number 
eigenvalues, and complex eigenvectors. 

Hermitian Inner Product 

The spectral theorem for symmetric matrices breaks down: 
15.9 Prove that the symmetric complex matrix 




is not diagonalizable. 

We will find a complex spectral theorem, but with a different concept 
replacing symmetric matrices. 

The equation \z\ 2 — zz is very important. Think of a complex number as if 
it were a vector in the plane: 



z = x + iy 




Then \z\ 2 = zz is the squared length x 2 + y 2 . 

The Hermitian inner product of two vectors z and w in C™ is the complex 
number 

(z, w) = ziwi + z 2 W2 + h z n w n . 

The curious bars on top of the w terms allow us to write ||z|| 2 = (z, z), just as 
we would for real vectors. Warning: the Hermitian inner product (z,u>) is a 
complex number, not the real number we had in inner products before. 

15.10 Compute (z, to) , ||z|| and for 

15.11 Prove that 

a. (w, z) = (z, w) 

b. (cz, w) — c (z, w) 

c. (z + w,u) — (z, u) + (w, u) 

d. (z,z) > 

e. (z, z) = just when z = 

for z, w and u any complex vectors in C™ and c and complex number. 



Adjoint of a Matrix 



The adjoint A* of a matrix A is the matrix whose entries are A*- = Aji (the 
conjugate of the transpose). Note that (A*)* = A. 

15.12 Prove that 

(Az,w) = (z,A*w) 

for any vectors z and w (if one side is defined, then they both are and they are 
equal) . 

15.13 Prove that if some matrices A and B satisfy (Az, w) = (z, Bw) for any 
vectors z and w for which this is defined, then B = A* . 

Self-adjoint Matrices 

A complex matrix A is self-adjoint if A = A* . This is the complex analogue of 
a symmetric matrix. Clearly self-adjoint matrices are square. 

15.14 Prove that sums and differences of self-adjoint matrices are self-adjoint, 
and that any real multiple of a self-adjoint matrix is self-adjoint. 

The matrices 




are self-adjoint. 

15.15 Which diagonal matrices are self-adjoint? 

15.16 Prove that the eigenvalues of a self-adjoint matrix are real numbers. 
Review problems 

15.17 A matrix A is called skew-adjoint if A* = —A. Prove that a matrix A is 
skew-adjoint just when iA is self-adjoint, and vice versa. 

Unitary Matrices 

A complex matrix A is unitary if A* = A^ 1 . This is the complex analogue of 
an orthogonal matrix. Clearly every unitary matrix is square. 

15.18 Prove that a matrix is unitary just when 

(Az,Aw) = (z,w) 

for any vectors z and w. 

15.19 Which diagonal matrices are unitary? 

15.20 If A is a real orthogonal matrix, prove that A is also unitary. 



Review problems 



15.21 Prove that the eigenvalues of a unitary matrix are complex numbers of 
modulus 1. 

Orthonormal Bases 

An orthonormal basis of C™ is a complex basis u\, 112, . ■ . , u n for which 



(u p ,u q ) = 



1 if p = q 
if p £ q. 



It may be helpful to use letters like p and q as subscripts, rather than i, j, to 
avoid confusion with the complex number i. 

15.22 Prove that any complex basis v\, V2, ■ ■ ■ , v n determines a orthonormal 
basis Ui,u 2 , ■ ■ ■ ,u n by the complex Gram-Schmidt process: 

Wp = V p (V p , U q ) U q 

q<p 



15.23 Apply the complex Gram-Schmidt process to find a orthonormal basis 
for the basis 

*"C) •*■(»)■ 

Normal Matrices 

A complex matrix A is normal if AA* = A* A. 

15.24 Prove that self-adjoint, skew-adjoint and unitary matrices are normal. 

15.25 Which diagonal matrices are normal? 

15.26 If A is normal, and c is a constant, prove that A + cl is also normal. 
Lemma 15.3. If A is normal, and Az = for some vector z, then A* z = 0. 

Proof. Suppose that Az = 0. Then 

0=\\Az\\\ 
= (Az, Az) , 
= (z,A*Az), 
= (z,AA*z), 
= (A*z,A*z), 
= \\A*zf. 



So A*z = 0. 



□ 



Lemma 15.4. If A is normal, then every eigenvector z of A with eigenvalue X 
is also an eigenvector of A* , but with eigenvalue X. 

Proof. Let B = A — XI. Then Bz = 0. Moreover, B is normal since A is. By 
the previous lemma, B*z = 0, so [A* — A/) z = 0. □ 



The Spectral Theorem for Normal Matrices 

At last, the complex version of our main theorem: normal matrices are diagonal 
after change of variables by a unitary matrix. 

Theorem 15.5. A square matrix is normal just when it is unitarily diagonal- 
izable. 



Proof. Let A be unitarily diagonalizable. So F*AF = A is diagonal, for some 
unitary matrix F. Then A = FAF*, and one easily checks that AA* = A* A. 

Let A be a normal matrix. Pick any eigenvector u\ of A, say with eigenvalue 
A. Scale U\ to be a unit vector. Pick unit vectors u 2 , 1*3, . . . , it„ so that 
iti, 112, U3, . . . , u n is an orthonormal basis. Let F be the associated unitary 
change of basis matrix, 



F 



(ui 



"2 



Replace A by F*AF. After replacement A is still normal, and Ae\ = Ae^ the 
first column of A is Aei. So 



A = 



for some smaller matrices B and C . By lemma 15.4, A*e\ — Aei, so the first 
column of A* is Aei, and so B — 0. Moreover, C is also normal, so by induction 
we can unitarily diagonalize C, and therefore A. □ 



Corollary 15.6. Self-adjoint, skew-adjoint and unitary matrices are unitarily 
diagonalizable. 

15.27 Let 




a. Is A self-adjoint or skew-adjoint? 

b. Find the eigenvalues and eigenvectors of A. 

c. Find a unitary matrix F which diagonalizes A, and unitarily diagonalize 
A. 



Appendix: The Fundamental Theorem of Algebra 



There was one missing ingredient in the proof of the complex spectral theorem: 
we need to know that complex polynomials have zeros. This gap is filled in here. 
Warning: students are not required or expected to work through this appendix, 
which is advanced and is included only for completeness. 

Lemma 15.7. Take 

p(z) = a + a\Z + • • • + a n z n 

any nonconstant polynomial. In order to keep p(z) at as large a modulus as we 
like, we only have to keep z at a large enough modulus. 

Proof. We can assume that a n ^ 0. Write 

p(z) a ax 

= — H -r H ho„. 

z n z n z n 

All of the terms get as small as we like, for z of large modulus, except the last 
one. So for large enough z, p{z)/z n is close to a n . Since z n has large modulus, 
p(z) must as well. □ 

Corollary 15.8. For any polynomial p{z), there must be a point z — z at 
which p(z) has smallest modulus. 

Proof. By lemma 15.7, if we choose a large enough disk containing zq then p(z) 
has large modulus at each point z around the edge of that disk. Making the disk 
even larger if need be, we can ensure that the modulus all around the edge is 
larger than at some chosen point inside the disk. By theorem 14.9 on page 132, 
there is a point of the disk where p{z) has minimum modulus among all points 
of that disk. The minimum can't be on the edge. Moreover, the modulus stays 
large as we move past the edge. Thus any minimum modulus point in that 
large disk is a minimum modulus point among all points of the plane. □ 

Lemma 15.9. The modulus \p{z)\ of any nonconstant polynomial function 
reaches a minimum just where p(z) reaches zero. 

Proof. Take any point zq. Suppose that p(zq) ^ 0, and lets find a reason why 
zq is not a minimum modulus point. Replace p{z) by p (z — zq) if needed, to 
arrange that z — 0. Write out 

p(z) — a + a\z + a 2 z 2 + • • ■ + a n z n . 

It might happen that a\ — 0, and maybe a 2 too. So write 

p(z) — a Q + a k z k H h a n z n , 



writing down only the nonzero terms, in increasing order of their power of z. 



Clearly ao ^ because p(0) ^ 0. We can divide by ao if we wish, which 
alters modulus only by a positive factor, so lets assume that ao = 1. We can 
rotate the z variable, and rescale it, which rotates and scales each coefficient. 
Thereby arrange a k = — 1, so 

p(z) = 1 - z k + --- + a n z n . 

Calculate 

bO)l 2 = p{z)p{z) 

= 1 - z k - z k + . . . , 

where the dots indicate terms involving more z and z factors. Write z = 
rcosQ + ir sin 9. De Moivre's theorem gives 

\p{z)\ 2 = 1 - 2 r k cos k9 + r k+1 (...). 

The error term (...) is some (probably very complicated) polynomial in r with 
(complicated) coefficients involving cos 9 and sin 9. We don't need to work it 
out. We only need to know that it is bounded for z near enough to 0, which is 
clear whatever the terms involved are. For r > sufficiently small, 

2-r(. ..) > 0. 

Multiplying by — r k , 

-2r k + r k+1 (...) < 0. 
Therefore |p(^)| 2 gets even smaller at the point z = r than at z = 0. □ 



Corollary 15.10. Every nonconstant complex polynomial has a root. 

Theorem 15.11 (Fundamental Theorem of Algebra). Every nonconstant com- 
plex polynomial p(z) can be factored into linear factors. More specifically, 

p{z)=c{z- Zl ) d i (z-z 2 ) d2 ...(z-z k ) d * 

where c is a constant, z\, z%, . . . , Zk are the roots of p(z), and d\, ■ ■ ■ , dk are 
positive integers, with sum d\ + d 2 + ■ ■ ■ + dk equal to the degree ofp(z). 



Proof. We have a root, say z\. Therefore p(z) / (z — z\) is a polynomial, and 
we apply induction. □ 
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1.5. Forward eliminate: 
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Scale row 3 by — 1. 
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Scale row 2 by — |. 



Add —(row 2) to row 1. 



Scale row 1 by — 1. 
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There are no solutions. 
1.18. Forward eliminate: 
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xi =0 
•x 2 =0 
x 3 = 
2:4 = 



1.21. You could try: 

a. One solution: x\ = 0. 

b. No solutions: x\ = 1, xi = 1, x\ + x-i = 0. 

c. Infinitely many solutions: X\ + Xi = 0. 
1.23. 




1.28. (a)=(2), (b)=(4), (c)=(l), (d)=(5), (e)=(3) 
2.2. 

(a) All coordinates of each vertex are ±1. 

(b) The vertices of a regular octahedron lie in the centers of the faces of a 
cube. 

(c) Try an equilateral triangle in the plane first. This should lead you to the 
points 




with an even number of minus signs. 



2.3. 



( * 



2.4. 



A = 



,B = 



2.9. Any matrix full of zeros with at least two columns. 

2.13. 1 -8 + (-1) ■ 1 + 2-3 = 13 

2.14. 



2.16. 



AB 



AC = 



2.18. You could try 



AD 



BC 



CA = 



A = 



2.20. In an upper triangular matrix, Ay = if i > j. So nonzero terms have 
i < j. The product: (AB)^ = J^k ^ikBkj The A^ vanishes unless i < k, and 
the second vanishes unless k < j , so the whole sum consists in terms with 
i < k < j. The sum vanishes unless i < j, hence upper triangular. Moreover, 
the terms with i = j must have i < k < j, so just the one AuBu term. 



2.22. 

{c{AB)) i3 - c{AB) %] 

k 

= ^ cAikBkj 
k 

fe 

= ((cA)B) ti . 

2.23. 

k 

= ^ ^ A it Bi k Ckj ■ 
k i 

On the other hand, 

(A(2?C)) - = $> ifc (BC) kj 

k 

= ^ ^ AikBkeCij . 
k i 

Since k and £ are just used to add up, we can change their names to anything 
we like. In particular, the resulting sums won't change if we rename A; to I and 
£ to k. Moreover, we can carry out the sums in any order. (You still have to 
show that each side is defined just when the other is.) 
2.24. 

(A(B + C)),^^^ {B + C) k] 

k 

= ^ Aik (Bkj + Ckj) 

k 

— ^ AikBkj + ^ AikCkj 

k k 

= (AB) ij + (AC) ij . 

3.2. One proof: 

k 

= Aij 

because In, — 1 just when k = i. 



A hint for a different proof (without using J^): if A is 1 X 1, then the result 
is clear. Now suppose that we have proven the result already for all matrices of 
some size smaller than p x q, but that we face a matrix A which is p x q. Then 
split A into blocks, in any way you like, say as 



A 



and write out 



I A 




and calculate out the result, using the fact that since P,Q,R and S are smaller 
matrices, we can pretend that have already checked the result for them. 



3.3. 
3.4. 
3.6. 



IB = BI = I but IB = B. 
IB = BI = I but IB = B. 



ei 



3.7. Row j. All rows except row j. 

3.8. One proof: 



{A n 

A 21 

\A nl 



A 12 
A 22 

A n2 



A ln \ 



Ao 



A nn J \0j 



so running your fingers along the rows of A and column of e\\ 



f A u -l + A 12 -0 + --- + A ln -0\ 
An ■ 1 + A 22 ■ + • ■ ■ + A 2n ■ 



\A nl ■ 1 + A n2 ■ + • • • + A nn ■ 0/ 

(A n \ 
A 2 i 

\A n J 

Another proof: e\ has entries (ei)^ = 1 if i = 1 and (ei)^ = if i ^ 1. So Ae\ 
has entries {Ae\) i — ^ fe Aik {&i) k - Each term is zero except if k = 1, in which 
case it is An. But the entries of the first column of A are A\x,A 2 x, . . . , A n \. 



3.10. Write x — J2 x j e j' an d multiply both sides by A. 

3.11. Use the fact that the columns of AB are A times columns of B. 

3.12. You could try 



A 



(/a 0) 



, B 



3.16. 
























































3.17. Mostly yes. You only have to try to figure out where e\ goes to, and 
where e-i goes to. The vector e\ is close to her left eye. The vector e<i is close to 
the top of her head, which is not really marked by anything, so harder to follow. 
But you can't figure out what matrix gives the straight line segment. Why? 
3.19. C = CI = C(AB) = (CA)B = IB = B. 
3.23. 

B~ 1 A~ 1 (AB) = B- 1 [A- 1 A) B 
= B~ X 1B 
= B~ l B 
= I. 



and similarly multiplying out (AB) B~ 1 A~ 1 . 

3.24. By definition, AA^ 1 = A~ X A = I. But these equations say exactly that 
A is the inverse of A^ 1 . 

3.25. Multiply both sides of the equation Ax — by A^ 1 . 

3.26. Multiply both sides of AB = I by A- 1 to find B = A' 1 . Therefore 
BA = I, and so A = B~ l . 

3.28. If x = y then clearly Ax — Ay. If Ax = Ay, then multiply both sides by 

A-K 

3.29. 



M~ 



A~ 








3.30. See figure 1 on the next page. 
3.31. 

5=1-7 






l\ 

10 0' 
1 0/ 

4.1. Each column has to be a column of the identity matrix, since it has all 
0's except for a single 1. But all of the columns have to be different, in order 
that no two columns have l's in the same row. So they are different columns 
of the identity matrix. If some column of the identity matrix doesn't show up 
anywhere in our matrix, say the third column for example, then the third row 
has only 0's. So every column of the identity matrix shows up, precisely once, 
scrambled in some permutation. 

4.4. Sej must be with multiples of rows added only to later rows, so column j 
of S has zeros above row j and 1 on row j. Hence S\j = S%j = ■ ■ ■ = Sj-ij = 
and Sjj = 1. 

4.5. Proof (1): Multiplying by R adds entries only to lower entries, preserving 
the l's on and 0's above the diagonal. 

Proof (2): A matrix R is strictly lower triangular just when Rij = unless 
i > j and Rij = 1 for i = j. 

(RS)^ = y^Rj k S kj . 

k 



3.32. 



P = 



/0 
1 




But this sum is all O's unless we find i > k and k > j, so we need i> j. If i = j, 
then only the term k = i = j makes a contribution, which is l>',,S,, = 1. 

Proof (3): Obvious for lxl matrices. Suppose that we have proven the 
result for all matrices of size smaller than n x n. If R and S are n x n, write 



R [a Bj ' S [p Qj 

where a and p are columns, and B and Q are strictly lower triangular. Then 

RS 



1 

a + Bp BQ 



which is strictly lower triangular because B and Q are strictly lower triangular 
of smaller size. 

4.6. Suppose that we want to make a matrix S which is p x q. Start with 
the identity matrix. Multiply it by the elementary matrix with Si q in row 1, 
column q. We get one element into place. Keep going, first getting things set 
up properly along the bottom row. 
4.8. 




4.10. If 



hi 



D 



ta 



t n J 



then 



\ 



If one of these t{ is 0, then there can't be an inverse, because Dei — 0, so if D 
has an inverse, then D~ 1 Dei = ej, but also D~ 1 Dei = D~ 1 = 0. 
4.12. 



' ad 











be 





V 





cf 



4.13. The original picture is 




The three resulting pictures are 
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4.14. 



4.15. P 3J = P swaps rows 1 and 2. 
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4.16. S 101 adds 202(row 1) to row 3. 



101 



/ 1 


\202 



4.24. Any number between and 3. 
4.25. 
a. 

A 2 

\0 



Impossible: can only use 1 pivot in each row, so at most 3 pivots binding 
up variables, so must have at least 2 left over free variables (or at least 
one free variable, if the last column is a column of constants). 



4.28. There is at most one pivot in each row. There are more columns than 
rows. So there must be a pivotless column: a free variable for the equation 
Ax = 0. 



5.1. Forward eliminate: 
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Back substitute: 



Add —(row 3) to row 2. 
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5.2. Forward eliminate: 
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Swap rows 2 and 3. 
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-1 


1 





Back substitute 

















Scale row 3 by — |. 





( H 


2 


3 


1 















-3 


-1 





1 




V o 





1 


1 

3 


_ 1 

3 





Add — 3(row 3) to row 1, 3(row 


3) to row 2. 








( E 


2 








1 


















— 1 


1 




V o 





1 


1 

3 


_ 1 

3 





Scale row 2 by 


i 

2 • 














( E 


2 








1 










1 








1 

2 


_ 1 

2 




V o 





1 


1 

3 


1 

3 





Add — 2(row 2) to row 1. 














f E 
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1 
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1 

2 




V o 





1 


1 

3 


1 
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5.3. Not invertible. 












5.4. 






















-1 








A~ 






1 














Q 







5.6. Yes, invertible. 

5.7. No, not invertible. 

5.8. If A is invertible, A~ 



Ax 



= 0. On the other hand, if Ax = holds 
only for x = 0, then the same is true of Ux = 0, after Gauss- Jordan elimination. 
Any column of U with no pivot gives a free variable, so there must be a pivot 
in each column, going straight down the diagonal, so U is invertible. 
5.9. Lets use theorem 5.4. Is there a solution x to the equation Ax — b for 
every choice of 6? Yes: try x — Bb. So A is invertible. Multiply AB — I on 
both sides by A^ 1 to get B = A^ 1 . 



5.10. We have already seen that if A and B are both invertible, then 
{ABy 1 = B- X A- X . Suppose that AB is invertible. Then (AB) (AB) _1 = I 

so A (B (AB)" 1 ) = I. Therefore A is invertible. Multiply on the left by A" 1 : 

B (AB)^ 1 = A- 1 . Multiply by A on the right: B(ABy 1 A = I. So B is 
invertible. 

5.11. Yes 

5.12. Forward elimination: 

2 
2 
1 

Add — l(row 1) to row 2. 



Swap rows 2 and 3 





2 


3 


o 


o 







1 


15 




2 


3 





1 


15 












2 


3 







The matrix is invertible, so the equations have a unique solution. 
5.15. You could try 



(0 



V 



which has forward elimination 

/l 









1 

1 

"2/ 



while its transpose A 1 has forward elimination 

2 

ED 





5.16. Ux — Vb just when V Ax — Vb just when Ax — b. So to solve Ax — b 
we need b to solve Ux — Vb. The last two rows of Ux = Vb give the conditions 
above. If those are satisfied, we can then drop those rows, and start solving the 
first two rows with the pivots. 

6.1. If a ^ 0, use it as a pivot, and forward elimination yields 

(a b 

be 



d- 

a 



Therefore if a ^ 0, then A is invertible just when d — — ^ 0. Multiplying by a, 
we see that A is invertible just when ad — be =/= 0. 

What if a = 0? We try to swap. Forward elimination yields 



Invertibility (when a = 0) is just precisely both b and c not vanishing. But 
(when a = 0) ad — be — —be vanishes just when b or c does, so just when 
invertibility fails. 
6.2. 

det I I = a det I I — c det 



6.3. 




ad — be. 



+3 (-3) -1(1) = -10 



6.4. 






1 det 




6.10. 1-4-6 = 24 

6.11. For lxl matrices U, the results are obvious: 



U=(u), U~ L = 



Suppose that U is n x n and assume that we have already checked that all 
smaller invertible upper triangular matrices. Split into blocks, in any manner 
at all 

o c) 



U 



with A and C square and upper triangular. You will have to find a way to see 
that A and C are invcrtible. Once you do that, check that 



U~ 1 = 



A~ 








We see by induction that (1) U^ 1 is upper triangular, (2) the diagonal entries 
of U^ 1 are the reciprocals of the diagonal entries of U, and (3) we can compute 
the entries U^ 1 inductively in terms of the entries of U. 

6.13. Swapping changes the sign of the determinant. But swapping doesn't 
change the matrix, so it doesn't change the sign of the determinant. Therefore 
the determinant is 0. 

6.14. You can use 



A : 



,B 



for example. 

6.15. The determinant goes up by 6 times. 

6.16. A 2 x 2 determinant looks like 



A\\Ai2 — Ai\A\2- 



7.1. 




Add —(row 1) to row 2, —(row 1) to row 3. 

/ 2 5 


\o i 



Make a new pivot 



Swap rows 2 and 3 



( 2 



\o 

( 2 





Make a new pivot \. 



/H 5 5\ 
6 



So det A — — (2)(1)(2): the minus sign because of one row swap. 

7.2. because the second row is a multiple of the first. 

7.3. From the fast formula, det A ^ just when there is a pivot in each column. 

7.4. Gauss- Jordan elimination with one row swap yields 








1 


-1\ 





H 


-1 


-1 








-1 


1 


u 








-1 / 



so 



h 





1 


-A 


i 














1 


-1 


-1 


V 





-1 


0/ 



det 



7.5. Gauss- Jordan elimination with one row swap yields 



/-l 







1 

3 
2 J 



det 



/o 



\2 





1 

1/ 



= G 



7.6. Gauss- Jordan elimination with no row swaps yields 



so 



det 



(2 


-1 


-A 





3 

_ 2 


i 

2 







0/ 


(2 


-1 




-1 


-1 


•) 


\2 


-1 





= 



7.7. 



7.8. 



7.9. 



-0 det 



-2 det 





- det 



2 det 





2 det 



det 





= -4 



-14 



+0 det 



1 



det 



1 







+ det 



7.10. Rescaling that row rescales the determinant. But rescaling that row 
doesn't change anything, so it must not change the determinant. The determi- 
nant must not change when scaled, so must be 0. 
7.12. To see that det A = 12, we compute 



Swap rows 1 and 2 



Add —(row 1) to row 3. 



/o 

3 



\3 


5 




1 


(! 


2 




5 




1 





2 


V° 


4 



Make a new pivot 



Add — 2(row 2) to row 3. 



/3 




Make a new pivot \. 



/m i 2 \ 

2 1 

V o o -2 ; 



7.13. If Ln = 0, then we have a zero row, so det = 0, and the result is obviously 
true. If Lu ^ 0, then use it as a pivot to kill everything underneath it: 



f L u 







L 





L 





L 



\ o 



£33 
£43 

L n 3 



Proceed by induction. 

7.15. Here is one proof: the i-th row of A 1 is obvious the transpose of the i-th 
column of A: A t — (Aei) . Writing out any vector x as x — X\e\ + X2&2 + 
• • • + x n e n , and adding up, we see that x t A t 
this to a column of B, say x — Bei. 



(Ax) for any vector x. Apply 



e i *B*A t = (Be i ) t A t 
= [ABeif 
= e^ABf. 



Here is another proof, using lots of indices: 



{AB)% = (AB) jl 

— 2^ AjkBki 

k 

= 2_j BkiAjk 
k 



7.16. It is the inverse permutation; see problem 4.2 on page 31. 

7.17. 4: expand down the third column. 



7.18. For k — 1, A 1 — A, so obvious. By induction, 



det (A k+1 ) 



= dot (A ■ A k ) 
= (det A) (det A k ) 
= (det A) (&etA) k 
= (detA) fc+1 . 



2222444466668888 



= (-1) 



2222444466668888 _ 



7.19. det (A 2222444466668888 ) = (det A) 
1 (an even number of minus signs) . 

7.20. AA- 1 = 1 so det Adet (A^ 1 ) = 1. 
7.21. 

a. By expanding down any column. 

b. By expanding across any row. 

c. By forward elimination, and then taking the product of the diagonal entries. 
(The fastest way for a big matrix.) 

7.22. One, because the determinant of the coefficients is 1 • 2 • 3 — 6. 
8.1. 

xi + 2x 2 + x 3 — 1 

%2 + X 3 = 

—xi + x 2 = -2 

8.3. Yes. 

8.4. Lets call these vectors x\, X2,x%. Make the matrix 



A = 



Xi 



•X'2 



3'3 



Apply forward elimination: 



.4 



Add — l(row 1) to row 3. 



/I 





1 

-V 



Swap rows 2 and 3 





1 


(: 


pr 









1 





pr 


\o 






If we add any vector y, we can't add another pivot, so every vector y is a linear 
combination of X\, X2, X3. Therefore the span is all of M 3 . 
8.5. Yes. Forward eliminate: 



Swap rows 1 and 2. 



/ 



Move the pivot 



Add —(row 2) to row 3. 



Move the pivot 



-1 


1 


-1 







2 


-1 


v) 





2 


1 






1 


-1 








2 


-1 


-1 





2 


1 


1 


pi] 


1 


-1 








2 


-1 


-1 








2 


2 


-1 


1 


-1 








2 


-1 


-1 








2 


2 



There is no pivot in the final column, so the final column is a linear combination 

of earlier columns. 

8.6. No. Forward eliminate: 



Move the pivot 



Add row 2 to row 3. 



Move the pivot 



Move the pivot 



( 


2 


4 





-1 







1 





1 


2 


4 





-1 





\o 


1 





LzJ 


2 


4 


n 
u 


— i 


n 
u 


i n 


n 

u 


n 


( 2 


2 


4 





-1 





\o 








( 2 


2 


4 





-1 





\o 









There is a pivot in the final column, so the final column is linearly independent 
of earlier columns. 
8.9. xi+x 2 + 2x3 = 

8.11. You can rescale and add as many times as you need to, forming any 
linear combination. 

8.12. No: it doesn't contain 0. 

8.13. Yes 

8.14. Obviously every vector in a subspace is a linear combination of vectors 
in the subspace: x = 1 • x. So the subspace lies inside the span of its vectors. 
Conversely, every linear combination of vectors in a subspace belongs to the 



subspace, so the span of the vectors in the subspace lies in the subspace. 
Therefore a subspace is its own span. 
8.15. {0} and R. 
8.16. 

a. Not always. Take the x and y axes in the (x, y) plane. 

b. Yes. 

8.17. No: it contains 



but doesn't contain 



2x = 



8.18. 

a. no 



b. 
c. 
d. 



yes 
yes 
no 



8.20. The lines through 0. 

9.2. Put the standard basis into the columns of a matrix, and you have the 
identity matrix. Look: there is a pivot in each column. 

9.4. Try adding a vector to the set. If you can't then you are done: a basis. If 
you can, then keep going. If you end up with more than n vectors, then use 
theorem 9.2. 

9.5. Put them into the columns of matrix A. You find det A — 0, so these 
vectors are linearly dependent. 

9.6. Put them into the columns of a matrix, and apply forward elimination to 
find pivots: 

(2 1 

A square matrix, a pivot in every column, so a basis. 
9.10. Write A as columns 



.4 



(ui 



U 2 



The equation Ax — is the equation x\U\ 
imposes a linear relation. Therefore u\,U2, ■ . . 
when Ax — has x — as its only solution. 
9.11. 

(\ -i N 



- X2U1 + • • • + x n u n = 0, which 
, u n are linearly independent just 



F~ 1 = 







F~ 1 AF = 







9.12. 



A = 



F~ 



(l 










1 


3- 











-2 


4 


(: 


1 












1 

r Mr = | o l 

,o o 



x q e q to give 



9.14. Expand out x — x%ei + x 2 e 2 + ■ 

Ax = xi (Aei) + x 2 (Ae 2 ) H hi, (Ae q ) . 

So Ax = just when x\, X2, ■ ■ ■ ,x q give a linear relation among the columns of 
A. 

9.15. No 
9.17. Yes 
9.19. Let 



F = 
G = 



(xi 



X 2 
!J2 



y, 



and let A — G F . But why is there only one such matrix? 

9.21. The idea is that e 2 — 63 = (ei — 63) — (ei — e 2 ), etc. So consider the 
vectors e\ — e 2l e\ — ea, . . . , e± — e n -\- Clearly if i = 1, then e, — ej is one of 
these vectors. But if i ^ 1, then 

e t - ej = (-1) (ei - ei) + (ei - e } ) . 

So the vectors e\ — e 2 , e\ — e^, . . . , e\ — e n _i span the subspace. Clearly these 
vectors are linearly independent, because each one has a nonzero entry just at 
a spot where all of the others have a zero entry. Alternatively, to see linear 
independence, any linear relation among them: 

= c 2 (ei - e 2 ) + c 3 (ei - e 3 ) + . . . c„ (e x - e„) 
= (c 2 + c 3 H h c„) e x + c 2 e 2 + c 3 e 3 H h c„e„ 

determines a linear relation among the standard basis vectors, forcing = 
c 2 = C3 = • • • = &„,. The subspace is actually the set of vectors x for which 
xi + x 2 + ■ ■ ■ + x n = 0. 

9.22. You could try 

/0 1 N 
A= 1 1 

\1 2; 



Depending on whether you swap row 1 with row 2 or with row 3, forward 
elimination yields 









m 




(: 









!) 













10.3. The reduced echelon form is 

/I 


\o 

The basis for the kernel is 




1 


1 1 

-2 

V 1 



10.6. Remove zero rows. 



(X 







1 



Change signs of the entries after each pivot. 






-2 



Pad with rows from the identity matrix, to get l's down the diagonal. 



/I 





Vo 






-2 
1 




-1 

-2 


1/ 



Keep the pivotless columns. 





-2 
1 

Vo/ 



-i 

-2 


v i / 



10.7. Remove zero rows. 

/0 |T| 

|T| 



\0 

Change signs of the entries after each pivot. 



/0 








1/ 








Pad with rows from the identity matrix, to get l's down the diagonal. 



(I 

o [T] 





Vo o 

Keep the pivotless columns. 



10.8. The reduced echelon form is 



(\ 





Remove zero rows. 



/l 




Vo 





1 






1 









1 




A\ 





Vo/ 




1 






1 





Change signs of the entries after each pivot. 



/I 





Vo 



o\ 





1/ 



2\ 

_ 5 
2 

5 
2 

1 / 



2\ 

_5 
2 

5 
2 

1 / 



5 
2 

_5 
2 

-1/ 



Pad with rows from the identity matrix, to get l's down the diagonal. 

"2\ 



/ 1 

o [T] 





V o o 

Keep the pivotless columns. 



5 
2 
_ 5 
2 

-1 
1 / 



10.9. 



10.10. The only vector in the kernel is 0. 

10.11. 1, 1, 2, 1, 1, 

10.13. The kernel of B consists in the vectors 



for which Bv = 0. This is just asking for Ax + Ay + Az = 0, i.e. for 
A(x + y + z) = 0. We can pick x and y arbitrarily, and pick an arbitrary vector 
w in the kernel of A, and set z = w — (x + y). In particular, B has kernel of 
dimension 2q + k where k is the dimension of the kernel of A. 
10.14. 

y = Ax 



3 

= x i Ae 



so a linear combination (with coefficients x±,X2, ■ ■ ■ 5 x q ) of the columns Aej of A. 
Therefore the vectors of the form y — Ax are precisely the linear combinations 
of the columns of A. 

10.16. It is the horizontal plane in K 3 , the xy-pl&ne in xyz coordinates. 
10.18. If A is tall, then A* is short, so there are nonzero vectors c so that 
A t c = 0, i.e. c f A = 0. Since c ^ 0, there must be some entry Cj ^ 0. If Ax = e^, 
we find = c Ax — c l ei — Ci a contradiction. So is not in the image. 



10.19. Forward eliminate: 





-1 





-1 







2 


2 




1) to 


row 3. 




H 


-1 








-1 







4 





Move the pivot 



Add 4(row 2) to row 3. 



Move the pivot 



(\ 




/l 





1 

I— 1 





~-\\ 




4 




-1 





~-\\ 









-1 






-1 





-1 

-5 



The pivot columns are columns 1,2,3. So the columns: 



form a basis. 

10.20. Forward eliminate: 









(-')• 


!■)■ 











( 2 

-6 



-G 
18 




Add 3(row 1) to row 2. 



Move the pivot \. 



Move the pivot — K 



Move the pivot — K 



f 2 


-6 


-6 











\o 









( 2 


-G 


-6\ 











V° 





0/ 


( 2 


-6 










o 


V° 







f 2 


-6 












v° 








The only pivot column is column 1. So the column: 

(2 

-6 



is a basis. 

10.21. Forward eliminate: 



Swap rows 1 and 2. 



( 


2 


2 


-1 


\i 


1 


( 2 


-1 





2 


V 1 


1 



Add — 5 (row 1) to row 3. 



Move the pivot \. 



( 2 





( 2 





Add — | (row 2) to row 3. 



Move the pivot 



( 2 


-1 





@ 








( 2 


-1 


2 





2 










1 



The pivot columns are columns 1,2,3. So the columns: 



form a basis. 

10.22. Forward eliminate: 



Add 3(row 1) to row 2. 





( 2 




-i 


W 


V 1 



( 2 





f 2 


-1 


3 


-6 


3 




\ 








Move the pivot \. 



Move the pivot — K 



Move the pivot — K 



f 2 


-1 


3 \ 


o 


o 





V u 


u 


n / 


( 2 


-1 










o 


V° 







f 2 


-1 




















The only pivot column is column 1. So the column: 

(2 

-6 



is a basis. 

10.23. Forward eliminate: 



/-3 


1 


3 


-1 


V 3 


-1 



Add row 1 to row 2, row 1 to row 3. 

/-3 


V o 



Move the pivot \. 



/-3 


V o 



Move the pivot — K 



Move the pivot 



-3 


1 











o 














V o 



The only pivot column is column 1. So the column: 

/-3 N 
3 

\3 

is a basis. 

10.24. Forward eliminate: 



( 2 


4 0\ 


-6 


-12 


V 4 


8 Oy 



Add 3(row 1) to row 2, — 2(row 1) to row 3. 



( 2 


4 








\o 






Move the pivot 



( 2 





Move the pivot 



( 2 


4 








V° 






Move the pivot — >. 



/ 2 4 (A 


v 0/ 



The only pivot column is column 1. So the column: 

(2 

-6 



\4 



is a basis. 

10.25. Forward eliminate: 



/I 
1 

V- 1 



Add —(row 1) to row 2, row 1 to row 3. 



/I 





Move the pivot \. 



Add —(row 2) to row 3. 



Move the pivot 



2 

V 





-1 







1 


i) 




1 




/I 


-1 







1 


i) 









/I 


-1 




o 


1 




\o 








The pivot columns are columns 1,2,3. So the columns: 




form a basis. 

10.26. Forward eliminate: 



/El 
i 

v-i 



Add row 1 to row 2, —(row 1) to row 3. 



Move the pivot 



Swap rows 2 and 3. 



Move the pivot 



/-l 


V o 



1 
1 

-1, 



1 ~\~ 













•J 


v 


2 




I— 1 





1 





[JL 


2 


, o 


2 


-2 







1 





2 


-2 


o 





2 



The pivot columns are columns 1,2,3. So the columns: 

/-1 N 
1 

V-L 




i 
i 



form a basis. 

10.27. Forward eliminate: 



-1 


4 




3 


-12 


-12 


3 


-12 


-12 



Add 3(row 1) to row 2, 3(row 1) to row 3. 



Move the pivot 



Move the pivot 



Move the pivot — K 



-1 


4 


4 





















4 


4 



















4 


4 




















4 










: ) 








0/ 



The only pivot column is column 1. So the column: 




is a basis. 

10.28. Forward eliminate: 

( ° 2 ^ 

2 2 

\0 2 Oy 



Swap rows 1 and 2. 



Move the pivot 



Add —(row 2) to row 3. 



Move the pivot 



(2 





(2 





(2 




(2 






1 





1 

-1 



The pivot columns are columns 1,2,3. So the columns: 




form a basis. 

10.29. Forward eliminate: 



( 2 


-6 




-6 


18 


12 


v° 









Add 3(row 1) to row 2. 



( 2 


-6 


-4 










V° 








Move the pivot \. 



Move the pivot — K 



Move the pivot — K 



f 2 


-G 


-4 

















f 2 





-4\ 

o 

0/ 



/2 -6 


-4 







\ o o 





The only pivot column is column 1. So the column: 




is a basis. 

10.30. Forward eliminate: 

/Ho o\ 

2 -1 -1 

V 1 -i o/ 

Add — 2 (row 1) to row 2, —(row 1) to row 3. 

m o \ 

-1 -1 

Move the pivot \. 



(1 











-1 






-1 






Add —(row 2) to row 3. 



Move the pivot \. 








" 





-1 











l\ 










~-\ 




















W 





The pivot columns are columns 1,2,3. So the columns: 



, 

form a basis. 

10.31. Ax — implies that Bx — CAx — 0. Conversely, Bx = implies that 
Ax = C~ l Bx = 0. 

10.32. Linear relations among vectors pass through C and through C _1 . 

10.33. Compute echelon form: 



Swap rows 1 and 2 



(° 


2 


2 


2^ 




1 


2 













1 


2 


2 






2 


2 






/I 


2 












2 


2 


2 







1 


2 


2 






2 


2 






/l 


2 





<A 





2 


2 


2 





1 


2 


2 


\o 


2 


2 


2/ 



Add -l/2(row 2) to row 3. 



Add — l(row 2) to row 4. 






Vo 



2 
1 

0/ 



So the rank is 3, the number of pivots. There is 1 pivotless column. The image 
is 3-dimensional, while the kernel is 1-dimensional. 
10.34. You could try 



A = 



B = 



, C- 



The image of A is the span of the columns, so the span 



of 



while the image of B is the span of its columns, so the span of 



which are clearly not the same subspaces, but both one dimensional. 
10.35. The rank of A 1 is the number of linearly independent columns of A* 
(rows of A) . Let U be the forward elimination of A. When we compute forward 
elimination, we add rows to other rows, and swap rows, so the rows of U are 
linear combinations of the rows of A, and vice versa. Thus the rank of A 1 is 
the number of linearly independent rows of U. None of the zero rows of U 
count toward this number, while pivot rows are clearly linearly independent. 
Therefore the rank of A* is the number of pivots, so equal to the rank of A. 
10.42. If they both have solutions, then A t y = so y t A = 0, so y t Ax = 0. But 
y 1 Ax = y t b, so b f y = 0, a contradiction. 

On the other hand, if neither has a solution, then b is not in the image of A. 
So b is not a linear combination of the columns of A, and the matrix M = ( A b) 
has rank 1 higher than the matrix A. Therefore the matrix 



M* = 



A* 



also has rank one higher than A 1 . So the dimension of the kernel of M f is one 
lower than the dimension of the kernel of A* , and therefore there is a vector y 
in the kernel of A* but not in the kernel of M* . 
10.45. 



A + B 







so you can use the previous exercise. 

10.46. (a) only 

10.47. The rank of AB is the dimension of the image. But (AB)x = A(Bx), 
so every vector in the image of AB lies in the image of A. The clever bit: on 
the other hand, the rank of AB is the same as the rank of AB , as shown in 
problem 10.35 on page 89. But AB* = B t A t , so the rank is at most the rank of 
£?*, which is the rank of B. 

11.2. The determinant det (A — XI) is the determinant of an upper (lower) 
triangular matrix if A is upper (lower) triangular, with diagonal entries. 

11.3. You could try 

,i o) 



.4 




which has eigenvalues A = 1 and A = — 1. 
11.4. You could try 

A= 1 ),B=( 1 



Their eigenvalues are: for A, A = 1; for B, X = 1, for A + B, A = 1 or A = 3. 

11.5. Suppose that A is an n x n matrix. The determinant of a matrix A is a 
sum of terms, each one linear in any of the entries of A which appear in it. The 
characteristic polynomial det (A — XI) therefore involves at most n terms with 
a A in them, coming from the n diagonal entries of A — A, so a polynomial in A 
of degree at most n. 

11.6. (-l) n A™ 

11.7. A : 0,5, B : -1,3, C : 0, 1, D : 0,0,2 

11.8. det A — det A 1 for any square matrix A, so J* = I and det (A — XI) = 
det (A* -XI). 

11.9. 

det (F~ 1 AF -XI)= det (F^ 1 (A -XI) F) 

= det (F- 1 ) det (A — XI) det (F) 
= det (A — XI) . 

11.11. If A is invertible, then 

det (AB - XI) = det (A" 1 (AB -XI) A) 
= det (BA — XI) . 

If B is invertible, the same trick works. But if neither A nor B is invertible, we 
have to work harder. Pick a any number which is not an eigenvalue of A. Then 
A — a I is invertible, so (A — a I) B and B (A — a I) have the same eigenvalues. 
Therefore 

det ((A — a I) B - X I) = det (B (A — a I) — XI) 



as polynomials in a. Now we can plug in a = 0. 



11.13. 0,3 
11.16. 

a. s n (A) = 1 

b. s (A) = det (A - 0) = det A 

c. The expression det A is a sum ofterms, each a product of precisely n entries 
of A, as we have seen. So det (A — XI) is a sum of terms, each involving 
some A entries and some A's, with a total of n factors in each term. 

d. If A is upper triangular, or lower triangular, then the result is obvious. 
But each term of s„_i(A) involves precisely one entry of A, hence linear 
in those entries. So s n ~i(A) is a linear function of A, i.e. s n -i(A + B) — 
s„_i(A) + s n -i(B). We can write any matrix as a sum of lower triangular 
and an upper triangular. 

e. Follows immediately from problem 11.9. 

f. Clearly F~ 1 AFej = F~ 1 Auj = for j > r. So we get zeros just where we 
need them, to be able to write 




P must have rank r, since there is nowhere else to put the r pivots, so P 
is invertible. Since A has rank r, so must F AF. 



g 



det (A - X I) = det 




det(P 



XI) ( 



n—r 



h 



has no terms of degree less than n — r in A. 
You could try 




11.17. The A = 3-eigenvectors are multiples of 




11.20. 



A= 1 




11.21. 



A = -2 
A = 3 




11.22. 



A= 1 



A = 2 



A = 3 



11.27. Permutation matrices are orthogonal, so P f = P . Alternately: any 
permutation P must generate a finite number of permutations P, P 2 , P 3 , . . . , 
so eventually we must find a loop: P m = P™ for some integers m > n. Being 
invertible, we see that P k = 7 for some integer fc. 

(a) ±1 since P* = P _1 , so detP = detP* = g^p. Alternately: P fc = 7 so 
(detP) fc = 1 so detP = ±1. 

(b) The trace is the number of 1 's that stay on the diagonal when we permute 
the columns of 7 to make up P, i.e. the number of fixed columns. This 
number can be any value between 1 and n, except for n — 1, since you can 
cyclically permute any number of elements, not fixing any, except that you 
can't permute one element without fixing it. 

P* = P _1 so P is the transpose of a permutation matrix. Therefore the 
rows can be permuted to yield the identity matrix. So all pivots equal 1. 
Pick an eigenvector v so that Pv — Xv. But then (v,v) = (P t Pv,v) — 



(c) 
(d) 



(Pv,Pv) 
X k = 1, i 
11.28. 



A 2 
A = 



v,v) 
-- ±1. 



So A = ±1. Alternately: any eigenvalue will satisfy 



11.29. 



11.30. 



11.31. 



11.32. 



11.33. 



11.34. 



11.35. 



11.36. 



11.37. 



11.38. 



11.39. 



11.40. 



11.41. 




(-2 
\0 -2 

2 




f-3 

V - : 



-3 



11.42. 



. 



12.1. The kernel of any matrix is a subspace. 

12.2. Ax = Xx so 

ABx = BAx 
= BXx 
= XBx. 



12.3. Depending on the order in which you write down your eigenvalues, you 
could get: 



F 



F _1 AF 



X 
1 


D N 

1 



n N 



\0 



12.4. Again, it depends on the order you choose to write the eigenvalues, and 
the order in which you write down basis vectors for each eigenspace. You could 
get: 



A= -3 



A= -2 



A = 



F 



F~ X AF 





-2 








1 


-1 







1 




2 


2 




-1 


-1 




-2 




-3 










-2 


: ) 








0/ 



12.5. You could get 



A = 2 
















) 


(i 


I 




















j) 





















F 



F- 1 





i 


i 

2 




i 


1 

2 







1 




i 




( ; , ; 


3 






2 






-1 




(2 





' 





2 





V» 








F _1 AF = 



12.6. Suppose we have a matrix F for which 
F^AF = 



Xn 



V 



Call this diagonal matrix A. Therefore AF — FA. Lets check that the columns 
of F are eigenvectors. We need only see that FA is just F with columns scaled 
by the diagonal entries of A. 



12.7. You could try 



A = 



because it has only one eigenvector, 



x 



o r 
,o o, 



(up to rescaling), with eigenvalue A = 0. Therefore there is no basis of eigenvec- 
tors. 

12.9. You could get 

(v) 

- (1) 




Let A = F-^AF. So A = FAF' 1 . Clearly A 100000 = 1, so A 100000 

^y^lOOOOO^-l _ J 



12.11. 



.4 




12.12. 



Vl 



A = 1 



V 2 



V 3 





H 







F = 


i 

2 





] 






1 


c 






-1 




**- 




1 








1 








2 






(I 





o\ 







1 












y 



Consider the coordinates of the vector 



w 

Numbers of people can't be negative. Clearly the vector must be a linear 
combination of eigenvectors, with a positive coefficient for the A = 1 eigenvector: 



aivx + a 2 v 2 + a 3 v 3 , 



with ai > 0. Over time, the numbers develop according to 




aivi + a 2 v 2 + 



0-3^3- 



Since the other eigenvalues are smaller than 1, their powers become very small, 
and their components in the resulting vector gradually decay away. Thereforethe 
result becomes every closer to 



everybody dead. So everyone dies, in an exponential decay of population. (It 
should be obvious, because we didn't allow any births in our model.) 
12.15. 



det (A- XI) =A 2 + A 

=A (A+l) 



A- X- 

Gauss-Jordan elimination yields 



Remove zero rows. 



(' 



Change signs of the entries after each pivot. 

(a - 

Pad with rows from the identity matrix, to get l's down the diagonal. 



1. 



Keep the pivotless columns. 

A = 

A-X 

V° 

Gauss Jordan elimination yields 

(° 1 

\0 

Remove zero rows. 

(o S 

Change signs of the entries after each pivot. 

(o H 

Pad with rows from the identity matrix, to get l's down the diagonal. 

C i) 

Keep the pivotless columns. 



it 

- ft) 



12.16. 



det (A — XI) =X 2 -3A + 2 

= (A-l)(A-2) 



A = l 

'o o N 



A-X-- 

Gauss-Jordan elimination yields 



I 1, 







1 





Remove zero rows. 

(S >) 

Change signs of the entries after each pivot. 



1 -2J 

Pad with rows from the identity matrix, to get l's down the diagonal. 



Keep the pivotless columns. 





1) 


1 





A = 2 

A — X = 
Gauss-Jordan elimination yields 



f -i o N 



0' 

V °/ 

Remove zero rows. 

(E • 

Change signs of the entries after each pivot. 

(a o) 

Pad with rows from the identity matrix, to get l's down the diagonal. 







1 





Keep the pivotless columns. 

ft: 



A = 1 



-2 



A = 2 



12.17. 



det (A - XI) =A 2 + 3A 
=A (A + 3) 



A= -3 



A- A = 
Gauss-Jordan elimination yields 



Remove zero rows. 



3 1 
,3 0, 










o 



1 Oj 

Change signs of the entries after each pivot. 

(S ») 

Pad with rows from the identity matrix, to get l's down the diagonal. 



Keep the pivotless columns. 



[1] 0' 
1, 

ft) 



A = 



A — X = 
Gauss-Jordan elimination yields 



1 
,3 -3, 



, o o . 



Remove zero rows. 

(E 

Change signs of the entries after each pivot. 

(a 

Pad with rows from the identity matrix, to get l's down the diagonal. 





I) 


1 





Keep the pivotless columns. 

(£ 

A = -3 
A = 



12.18. 

det (A — XI) ~X 2 + 2 A 
=X (A + 2) 

A = -2 

-(-'. -') 

Gauss Jordan elimination yields 







1 





Remove zero rows. 

(s -0 

Change signs of the entries after each pivot. 



Pad with rows from the identity matrix, to get l's down the diagonal. 





I) 


1 





Keep the pivotless columns. 



A-X = 

Gauss Jordan elimination yields 



(!) 

-1 -1 N 










i 



Remove zero rows. 



( 



1 1 



Change signs of the entries after each pivot. 



-1 



Pad with rows from the identity matrix, to get l's down the diagonal. 













Keep the pivotless columns. 

it 



- 2 1 



A = 



-1 



12.19. 

det (A -XI) =A 2 -4 

= (A-2) (A + 2) 

A = -2 

— t -J' 

Gauss-Jordan elimination yields 
Remove zero rows. 

(a 

Change signs of the entries after each pivot. 

(a -i) 

Pad with rows from the identity matrix, to get l's down the diagonal. 













Keep the pivotless columns. 

(-.■: 

A = 2 

/ 4 



A- A = 
Gauss Jordan elimination yields 



-4 





2 





0/ 



Remove zero rows. 



(S ») 

Change signs of the entries after each pivot 

(S - 



Pad with rows from the identity matrix, to get l's down the diagonal. 



Keep the pivotless columns. 













-2' 



A = -2 
A = 2 



-1' 
- 2 ' 



12.20. 

det {A- XI) =A 2 +4A + 3 

= (A + 3) (A + l) 

A = -3 

— (-° 2 

Gauss-Jordan elimination yields 













Remove zero rows. 

Change signs of the entries after each pivot. 



1 1 

Pad with rows from the identity matrix, to get l's down the diagonal. 













Keep the pivotless columns. 



A = -1 

{-2 0, 

Gauss-Jordan elimination yields 










o 



Remove zero rows. 



( 



1 



Change signs of the entries after each pivot. 







Pad with rows from the identity matrix, to get l's down the diagonal. 













Keep the pivotless columns. 

c: 

A= -3 

A = -1 



12.21. 



det (A- XI) =A 2 + A 

=A (A+l) 



A= -1 



A-X 



1 
,0 1, 



Gauss-Jordan elimination yields 



[1] 
,0 



Remove zero rows. 

(o 

Change signs of the entries after each pivot. 



v° 1 

Pad with rows from the identity matrix, to get l's down the diagonal. 

(; :) 



Keep the pivotless columns. 

A = 

A-X = 
Gauss Jordan elimination yields 



-l v 

0, 







i 





Remove zero rows. 

(2 

Change signs of the entries after each pivot. 

(a 

Pad with rows from the identity matrix, to get l's down the diagonal. 





I) 


1 





Keep the pivotless columns. 




12.22. 

det (A — XI) =A 2 + A-2 

= (A + 2) (A-l) 

A = -2 

Gauss-Jordan elimination yields 







1 





Remove zero rows. 

(S 

Change signs of the entries after each pivot. 

(a 

Pad with rows from the identity matrix, to get l's down the diagonal. 

(i :) 

Keep the pivotlcss columns. 

C) 

A= 1 



A-\ = 
Gauss-Jordan elimination yields 



-1 -2, 



LU 2 
o o, 



Remove zero rows. 



( 



1 2 



Change signs of the entries after each pivot. 

(a -2) 

Pad with rows from the identity matrix, to get l's down the diagonal. 













Keep the pivotless columns. 

it 



A = 1 



-2' 




12.23. 



A = 1 



det (A- XI) =A 2 -3A + 2 

= (A-l)(A-2) 



A — X— (® 1 
\0 1 , 



Gauss Jordan elimination yields 



[1] 

,0 



Remove zero rows. 

(" E) 

Change signs of the entries after each pivot. 



(0 S) 



Pad with rows from the identity matrix, to get l's down the diagonal. 



1 



Keep the pivotless columns. 

A = 2 

A-X = 

Gauss Jordan elimination yields 



(ij 

. 










i 



Remove zero rows. 

(E 

Change signs of the entries after each pivot. 

(a -0 

Pad with rows from the identity matrix, to get l's down the diagonal. 







1 





Keep the pivotless columns. 

K 

X-l ( 




12.24. 

det (A — XI) =A 2 - 1 

= (A-1)(A + 1) 

A = -l 

Gauss Jordan elimination yields 

/ 1 



Remove zero rows. 







(a - 

Change signs of the entries after each pivot. 

03 

Pad with rows from the identity matrix, to get l's down the diagonal. 



1 


i 







Keep the pivotless columns. 

A= 1 

A-X = 

Gauss Jordan elimination yields 



2 -2' 
A -4, 



Remove zero rows. 



(: 


:) 









-0 



Change signs of the entries after each pivot 

(S 



Pad with rows from the identity matrix, to get l's down the diagonal. 



Keep the pivotless columns. 





I) 


1 





A = -1 



A = 1 



12.25. 



A = 







/ 





det (A - XI) =A 2 -A 

=A (A-l) 



A-X = 

Gauss-Jordan elimination yields 
Remove zero rows. 




-1 1, 







1 





(S 

Change signs of the entries after each pivot. 

(a 

Pad with rows from the identity matrix, to get l's down the diagonal. 



Keep the pivotless columns. 





I) 


1 





A = 1 



A-X = 

Gauss-Jordan elimination yields 



"I 0; 



[1] 0' 

Remove zero rows. 

/ 1 



Change signs of the entries after each pivot. 







Pad with rows from the identity matrix, to get l's down the diagonal. 













Keep the pivotless columns. 

ft: 



*- ft 



4- 

12.26. 



A = -3 



det {A- XI) =A 2 +4A + 3 

= (A + 3) (A + l) 



V 3 3/ 

Gauss Jordan elimination yields 

(l -2\ 
I 



Remove zero rows. 

f 1 -2 



Change signs of the entries after each pivot. 

(E >) 

Pad with rows from the identity matrix, to get l's down the diagonal. 







1 





Keep the pivotless columns. 



A = -1 

\ 3 3/ 

Gauss Jordan elimination yields 



i 






7 



Remove zero rows. 



I 1 

Change signs of the entries after each pivot. 

(a i) 

Pad with rows from the identity matrix, to get l's down the diagonal. 



Keep the pivotless columns. 



I— 1 


9 


V 





A = -3 
A = -l 



12.27. See table 1. 



Table 1: Invertibility criteria. A is n x n of rank r. U is any matrix 
obtained from A by forward elimination. 



§ 


Invertible 


Not invertible 


5 


Gauss- Jordan on A yields 1. 


Gauss- Jordan on A yields a 
matrix with n — r zero rows. 


5 


U is invertible. 


U has n — r zero rows. 


5 


Pivots lie on the diagonal. 


Some pivot lies above the di- 
agonal, and all pivots after it. 


5 


U has no zero rows 


U has n — r zero rows. 


5 


U has 71 pivots. 


U has t < 7i pivots. 


5 


Ax = b has a solution x for 


Ax = b has no solution for 




each b. 


some 6, n — v dimensions worth 
for other b. 


5 


Ax — b has exactly one solu- 


Ax = b has no solution for 




tion x for each b. 


some b, many for other b. 


5 


Ax — b has exactly one solu- 


Ax = b has no solution for 




tion x for some b. 


some b, many for other b. 


5 


Ax — only for x = 0. 


Ax = for many x. 


5 


^4 has rank n. 


A has rank r < n. 


7 


j4* is invertible. 


A* is not invertible. 


7 


detA^ 0. 


Every square block larger 
than r x r has det = 0. 


9 


The columns are linearly inde- 


The n — r pivotless columns 




pendent. 


are linear combinations of the 
r pivot columns. 


9 


The columns form a basis. 


Each of the n — r pivotless 
columns is a linear combina- 
tion of earlier pivot columns. 


9 


The rows form a basis. 


One row is a linear combina- 
tion of earlier rows. 


10 


The kernel of A is just the 


The kernel has positive dimen- 




vector. 


sion n — r. 


10 


The image of A is all of R". 


The image has positive dimen- 
sion r. 



§ Invertible 



Not invertible 



11 is not an eigenvalue of A. The A = eigenspace has pos- 

itive dimension n — r. 



13.1. 



13.2. 



13.3. 



{Aej,ei) = (A kj e k ,ei) 
= An. 



— ( e p(i)j e j) 

1, if p(i)=j, 
0, otherwise. 

1) if « =P~ 1 (j), 
0, otherwise. 

(eijfip-iy)) 

-l 



F ji 



k 
k 

13.4. 

/ (v,u) \ / \ / \ 

( V ttU, u ) = (v.u) q- (u, u) 

\ Nl / INI 

= (v,u) - (v,u) 
= 0. 

13.5. 

(a) One: x = 0. 

(b) 2n: one is ±1, all other xj are 0. 

(c) 2n + 16(2): one x, is ±2, all other xj are 0, or 4 Xj's are ±1 and all other 
are 0. 

(d) 2n + 8(n-2)Q)+2 6 (n-5)Q)+2 9 Q): 



a. one ±3 or 

b. two ±2's and one ±1 or 

c. one ±2 and five ±l's or 

d. nine ±l's. 

13.6. The hour hand starts at angle 7r/2, and completes a revolution every 12 



hours. So the hour hand is at an angle of 



7T 
2 



2tt 
— i 
12 



2nt. 



after t hours. The minute hand, if we measure time in hours, revolves every 
hour, so has angle 

-5 

a. t = jf(2k + 1), any integer k 

b. t = jf, any integer k. 
13.11. You could try: 



A = 



D = 



AB = 



13.12. For A symmetric. 

13.19. Those which have ±1 in each diagonal entry. 
13.20. 



P 



is not of that form. 

13.21. You could try A = B = I. 

13.22. 



J (i 



y\\ - IfII -llyl 



Preserve the right hand side, and you must preserve the left hand side. 

13.25. The rows of A are orthonormal just when A f is orthogonal, which occurs 
just when A — (A*) , which occurs just when A* = A -1 , which occurs just 
when A is orthogonal. 

13.26. See table 2 on the next page and table 3 on the following page. 
13.27. 



Ui 




u 2 = 




«:s = 




13.28. The pictures should look like 



Vl 

(l 



= V 2 — -, r Wl 



I 2 
= 

■ (i 



(2)(l) + (0)(-l) + (-2)(0) 
(1)(1) + (-1)(-1) + (0)(0) 



( 1 



Tabic 2: Orthogonalizing vectors: the projections 



U\ = Wi 



v/(l)(l) + (-l)(-l) + (0)(0) 



\y/2 
~hV2 



-1 

W 



U 2 





1 



y/(w 2 ,w 2 ) 



w 2 



v /(l)(l) + (l)(l) + (-2)(-2) 



/l 
1 

V 2 y 




Table 3: Orthogonalizing vectors: rcscaling 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 

Projected. 
Shrink/stretch all 
vectors to length 1. 




Done: orthonormal. 



13.29. 



ui 



l 



U2 



1 

\~7SJ 



Notice that Vi,V2,v^ did not give a basis, so when we try to compute u 3 , we 
run into trouble. 

13.30. The only problem that can come up is division by zero. But that 
happens only when we divide by a length If this length is 0, then Wj is 

zero, so 

i 

But each itj is a linear combination of vectors V\, V2, ■ ■ ■ , «j, so this is a linear 
dependence. 

13.32. If v is perpendicular to u, then set w — v, see that v is perpendicular 
to v, so v = 0. Otherwise, if v is not perpendicular to u, then w = v — u is 
perpendicular to u and v, and therefore perpendicular to any linear combination 
of u and v. In particular, since w is a linear combination of u and v, w must be 
perpendicular to w, so w = 0. So 



INI 2 '' 



13.33. Nothing. 



13.34. 



w\ = vi 



w 2 = v 2 



(V2,wi) 

(5)(l) + (3)(-l) 
(1)(1) + (-1)(-1) 



Ul 



: Wl 



Wi,Wi) 



U2 = 



V(i)(i) + (-i)(-i) 

m 

i 

1 /V 

v /(4)(4) + (4)(4) U, 



See figure 2 on the next page. 



The original vectors. 




Project the second 
vector perpendicular to 
the first. 




Projected. 
Shrink/stretch all 
vectors to length 1. 



Done: orthonormal. 



Figure 2: Orthogonalizing vectors in the plane 



13.35. 



W 1 = Vt 



■(::) 

(V2,Wi) 

W 2 = V 2 - ~, r Wi 

[Wl,Wi) 

= (o\ (Q)(-i) + (2)(-l) 
\2J (-l)(-l) + (-l)(-l) 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 



i 



Projected. 
Shrink/stretch all 
vectors to length 1. 



Done: orthonormal. 
Figure 3: Orthogonalizing vectors in the plane 



Ui = 



U2 



W 1 



1 



V(-i)(-i)+(-i)(-i) V 

m 

i 

V (w 2 ,w 2 ) 

v /(-i)(_i) + (i)(Ty ( i / 

hV2 



See figure 3. 



13.36. 



Wi = Vl 



- u) 

(V2,Wl) 
W2 = V 2 - ~, r W i 



-l) (-!)(-!) + (2)(1) /-l' 
2 1 (-!)(-!) + (!)(!) I 1 _ 



Ul 



u 2 



V(-i)(-i) + (i)(i) 



i 



1 



: W 2 



V(i)(i) + (5)(i) 




See figure 4 on the next page. 



\ 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 



Done: orthoriormal. 



Figure 4: Orthogonalizing vectors in the plane 



13.37. 



w 1 = v 1 



\ 1 / 

(V2, Wi) 



(wi,Wi) 

-A (-1)(2) + (!)(-!) ( 2 N 
1 ) (2) (2) + (-!)(-!) Ul, 



The original vectors. 



TO 

Project the second 
vector perpendicular to 
the first. 

Lin 

Projected. 
Shrink/stretch all 
vectors to length 1. 




Done: orthonormal. 



igure 5: Orthogonalizing vectors in the plane 



Ui 



U 2 



\f{Wl,Wx) 



Wl 



V(2)(2) + (-!)(-!) 



2 



4 A 



W 2 



jam +(!)(§) 
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13.38. 



tui = Vi 



w 2 = v 2 



\V2,Wi) 



{wi,wi) 

0\ (0)(l) + (2)(2) / 1 N 
2 (l)(l) + (2)(2) I 2 , 



1 

1 



"2 



V(l)(l) + (2)(2) V 2 . 



W2 




y/(w 2 ,W 2 ) 



v/H)H) + (§)(§) 




See figure 6 on the next page. 



L 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 



13.39. 



Done: orthonormal. 



Figure 6: Orthogonalizing vectors in the plane 



wi = v x 



W<2 = «2 ; r Wi 



{W 1 ,W 1 ) 

V\ (2)(1) + (0)(1) A N 

J (i)(i) + (i)(i) li 



l 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 



Done: orthonormal. 



Figure 7: Orthogonalizing vectors in the plane 



Ml = 



: Wl 



Wl,Wl) 



v / (l)(i) + (i)(l) V 1 , 



U2 = 



: W 2 



w 2 ,w 2 ) 



VOT) + (-!)(-!) 



13.40. 



W 2 = V 2 - -, r Wi 

{w 1 ,w 1 } 

(q\ (0)(1) + (2)(1) / 1 N 
\ 2 (1)(1) + (1)(1) ll 



Ml 



1 

1 



: Wl 



V(i)(i) + (i)(i) V 1 , 



■ i 



1 



u 2 = 



■ W 2 



W 2 ,W 2 ) 



V(-l)(-l)+(l)(l) 



-1^ 



See figure 8 on the next page. 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 



Done: orthoriormal. 



Figure 8: Orthogonalizing vectors in the plane 



13.41. 



Wl = Vi 



W 2 = V 2 



-A (-!)(!) + (2)(1) (l 
2 ) (1)(1) + (1)(1) \ Xj 

1) 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 











\ 











Done: orthonormal. 



Figure 9: Orthogonalizing vectors in the pi 



Ux 



: Wl 



V(l)(l) + (1)(1) U, 




ll 2 = 



VI 



: U> 2 



W 2 ,W 2 ) 



V(-i)(-§) + (§)(§) 



-1^ 



13.42. 



Wi = Vi 



w 2 = v 2 



[V2,Wl) 



U'l 



^ 1 / 



(-1)(Q) + (!)(!) /V 
(0)(0) + (l)(l) li 



Ul 



1 

1 



: Wl 



V(o)(o) + (i)(i) \i y 





U2 



■ W 2 



w 2l w 2 ) 



v /(-i)(-i) + (o)(o) \0 



See figure 10 on the next page. 



ft 
ft 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 



Li 

' I Done: orthonormal. 
Figure 10: Orthogonalizing vectors in the plane 



13.43. 



Wi = v 1 



w 2 = v 2 



[V 2 ,Wi) 



Wi 



(-1)(2) + (2)(1) /V 
(2)(2) + (l)(l) h, 



2 . 



The original vectors. 

Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 



Done: orthonormal. 



Figure 11: Orthogonaliziiig vectors in the plane 



u 2 



Wi,Wi 



1 2" 
V(2)(2) + (l)(l) {l y 

i 

V (w 2 ,w 2 ) 
1 

v /(-i)(_i) + (2)(2y 

IV5 



See figure 11. 



13.44. 



Wl = Vl 



\V2,Wl) 
W 2 = V 2 ~ 7 r Wi 



1 \ (!)(-!) + (-l)(0) (-1 
1 (-!)(-!) + (0)(0) 1 , 







Ux 



: Wl 



Wl,Wl) 



V(-i)(-i) + (o)(o) I o 







1 



u 2 = 



VI 



■ W 2 



w 2 ,w 2 ) 



^(o)(o) + (-i)(-i) V-i, 



See figure 12 on the next page. 




The original vectors. 




Project the second 
vector perpendicular to 
the first. 




Projected. 
Shrink/stretch all 
vectors to length 1. 




Done: orthonormal. 
Figure 12: Orthogonalizing vectors in the plane 



13.45. 



w 1 = v 1 



W 2 = V 2 ~ -, r W! 



(wi,Wl) 

0\ (Q)(2) + (-!)(-!) / 2 N 
-1 (2) (2) + (-!)(-!) Ul, 



The original vectors. 



1^ 



Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 



Done: orthonormal. 
Figure 13: Orthogonalizing vectors in the plane 



yj (wi,lUl) 



"2 



v /(2)(2) + (-l)(-l) \-\ t 



w 2 



VH)H) + H)H) 



5 V °/ 



See figure 13. 



13.46. 



Wi = v 1 

■C) 

W 2 =V 2 - -, r W\ 

{Wi,Wi) 



-l\ (-!)(!) + (0)(1) ( 1 
3 I (1)(1) + (1)(1) ll 



1 

Ml = = W\ 

1 



i 

M 2 = W 2 
1 



+ (§)(§) 



1^, 



See figure 14 on the next page. 
14.1. If Au = Xu and Av = (iv, then 



(Au, v) — X (u, v) 
= (u,Av) 
= V (u, v) 



2 / 



So (u,v) = 0. 



The original vectors. 



Project the second 
vector perpendicular to 
the first. 



Projected. 
Shrink/stretch all 
vectors to length 1. 



Done: orthonormal. 
Figure 14: Orthogonalizing vectors in the plane 




14.2. 

A = 16 
A = 3 

Gram-Schmidt the eigenvectors, and 

F: 
F- 1 : 



F~ 1 AF = 

Each eigenvector comes from a different eigenvalue, so they are already perpendicular- 

you only have to rescale them to have unit length. 

14.4. 

det (A — XI) =A 2 -5A 
=A (A - 5) 

A = 5 



A = 





3 




\/l3 




2 


VT3 


VIS 




3 


_ VTS 


Vl3 




2 






16 


) 


3 





After orthogonalizing the eigenvectors, 



F = 



F~ 1 = 





















1 








2 


V5 


75 




14.5. 



F~ X AF 



det (A — XI) =A 3 - 8 A 2 + 15 A 
=A (A - 3) (A — 5) 

>\ 

A = 3 



A = 



A = 5 



After orthogonalizing the eigenvectors, 



F 



14.6. 



F- 1 = 



F^AF = 



i 

2 





f 





1 


2 


V5 


75 


\ 75 


1 


/3 












det (A — XI) =A 3 - 4A 2 + 5A-2 
= (A-2) (A-l) 2 




After orthogonalizing the eigenvectors, 



14.7. 




After orthogonalizing the eigenvectors, 



F 



F^AF 



f-l 



VI 
f-i 



F~ = 



V! 
/-i 
o 





1 




1 




1 





14.8. Such an F must preserve the eigenspaces, so preserve the span of e\, the 
span of e2, and the span of e^. Therefore 



'±1 

F= I ±1 



±1. 



14.11. You only change X1X2 terms: 
2 

a. Xn 



b 



c. x\ + § + § £2X1 



14.12. 



A = 



A 



A = 



A -{1 «J 

14.14. Look at the eigenvalues of the symmetric matrix, to get started. 

a. ellipse 

b. hyperbola 

c. pair of lines 

d. hyperbola 
c. line 

f. empty set 

14.15. First, by orthogonal transformations, you can get you quadratic form 
to look like 

Ai x\ + \ 2 x\ -\ \- \ n x^. 

Then you can get every eigenvalue Xj to be 0, 1 or —1, by rescaling the associated 
variable Xj . Then you can permute the order of the variables. So you can get 

±xj + ±xl-\ \-±xl, 

for some s between 1 and n. 

15.1. Take the complex number with half as much argument, and square root 
as much modulus. 



15.10. (z, w) = + i(2 - 2i) = 2 + i 
15.16. Since A is self-adjoint, 

(Az, z) = (z, Az) . 

If we pick z an eigenvector, with eigenvalue A, then the left side becomes 

(Az,z) = A (z,z) , 

and the right side becomes 

(z, Az) = A (z, z) . 

Since (z,z) = \\z\\ 2 ^ 0, we find A = A. 

15.21. For an eigenvector z, with eigenvalue A, 

(z,z) = (Az,Az) 

(because A is unitary) 

= (Az, Az) 
= A A (z, z) 
= |A| 2 (z,z). 

Since z ^ 0, we can divide (z, z) off of both sides. 
15.23. 

ui -- 

15.27. 

a. A is self-adjoint. 

^ 3 (I) 

A = 4 

U(i-o, 







/l+2« 




, "2 = 


[ VlO 
1 2-i 


V2/ 







c. 



F = 
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List of Notation 



R" The space of all vectors with n real number 15 
entries 

p x q Matrix with p rows and q columns 15 

Any matrix whose entries are all zeroes 17 
S Sum 20 

1 The identity matrix 23 

I n The n x n identity matrix 23 

d The i-th standard basis vector (also the i-th 23 

column of the identity matrix) 

A" 1 The inverse of a square matrix A 25 

det The determinant of a square matrix 45 

A The transpose of a matrix A 55 

dim Dimension of a subspace 78 

kcr A The kernel of a matrix A 81 

im A The image of a matrix A 85 

{x, y) Inner product of two vectors 113 

1 1 a; 1 1 The length of a vector 113 

\z\ Modulus (length) of a complex number 134 

argz Argument (angle) of a complex number 134 

C The set of all complex numbers 134 

C" The set of all complex vectors with n entries 134 

(z,w) Hermitian inner product 136 

A* Adjoint of a complex matrix or complex linear 137 
map A 
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Index 



adjoint, 137 
argument, 1 34 

back substitution, 6 
ball, 131 

center, 131 

closed, 131 

open, 131 

radius, 131 
basis, 71, 76 

orthonormal, 116, 138 

standard, 72 
bounded, 131 
box 

closed, 131 

center 

of ball, see ball, center 
change 

of variables, 74 
change of basis, see matrix, change of 
basis 

characteristic polynomial, see polyno- 
mial, characteristic 

circle 

unit, 135 
closed 

ball, see ball, closed 

box, see box, closed 

set, 131 
combination 

linear, see linear combination 
complex number, 133 

imaginary part, 133 

real part, 133 
complex plane, 133 
conjugate, 1 34 
continuous, 130 
convergence, 130 

of points, 130 



de Moivre, see theorem, de Moivre 
decoupling 

theorem, see theorem, decoupling 
determinant, 45 
diagonal 

of a matrix, 1 6 
diagonalize, 104, 127 

orthogonally, 124 
dimension 

of subspace, 78 
dot product, see inner product 

echelon form, 16 

equation, 9 

reduced, 78 
eigenspace, 103 
eigenvalue, 93 

complex, 135 

multiplicity, 97 
eigenvector, 93 

complex, 135 
elimination, 3 

forward, 4 

Gauss- Jordan, 7 

fast formula 

for the determinant, 53 
Fibonacci, 24 

forward, see elimination, forward 
free variable, 8 

fundamental theorem of algebra, 14 1 

Gauss-Jordan, see elimination, Gauss- 
Jordan 

Hermitian 

inner product, see inner product, 
Hermitian 

identity matrix, see matrix, identity 
image, 85 
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independence 

linear, see linear independence 
inner product, 113 

Hermitian, 136 
inverse 

of a matrix, see matrix, inverse 

kernel, 81 
kill, 37, 81 

length, 113 
linear 

combination, 62 

equation, 3 

independence, 71 

relation, 71 

matrix, 15 

addition, 18 

change of basis, 73 

diagonal entries, 16 

identity, 23 

inverse, 25, 39 

multiplication, 20 

normal, 138 

self-adjoint, 137 

short, 37 

skew-adjoint, 137 

square, 15, 25 

strictly lower triangular, 31 

strictly upper triangular, 33 

subtraction, 18 

symmetric, 115 

unitary, 137 

upper triangular, 47 
minimum 

principle, 123 
modulus, 134 
multiplicity 

eigenvalue, see eigenvalue, multi- 
plicity 

negative definite, see quadratic form, 

negative definite 
normal matrix, see matrix, normal 

open 

ball, see ball, open 
orthogonal, 115 

orthogonally diagonalize, see diagonal- 
ize, orthogonally 



orthonormal basis, see basis, orthonor- 
mal 

perpendicular, 113 
pivot, 4, 16 

column, 64 
plane 

complex, see complex plane 
polar coordinates, 133 
polynomial 

characteristic, 93 
positive definite, see quadratic form, 

positive definite 
positive semidefinite, see quadratic form, 
positive semidefinite 

product 

dot, see inner product 

scalar, see inner product 
product, inner 

see inner product, 113 
Pythagorean theorem, see theorem, Pythagorean 

quadratic form, 125 

negative definite, 129 
positive definite, 129 
positive semidefinite, 129 

radius 

of ball, see ball, radius 
rank, 10 

reduced echelon form, see echelon form, 
reduced 

relation 

linear, see linear relation 
row echelon form, 9 
row operation, 28 

scalar product, see inner product 
self-adjoint, see matrix, self-adjoint 
short matrix, see matrix, short 
skew-adjoint matrix, see matrix, skew- 
adjoint 

spectral 

theorem, see theorem, spectral 
spectrum, 94 
square, see matrix, square 

matrix, see matrix, square 
standard basis, see basis, standard 
subspace, 66 
substitution 



back, see back substitution 
symmetric 

matrix, see matrix, symmetric 

theorem 

de Moivre, 134, 141 

decoupling, 104 

Pythagorean, 113 

spectral, 124 
trace, 97 
transpose, 31, 56 

unit circle 

see circle, unit, 135 
unit disk, 135 
unitary 

matrix, see matrix, unitary 

variable 

free, see free variable 
vector, 15 



